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ABSTRACT 


This  thesis  analyzed  the  use  of  a  multi-phase  source  selection  strategy  at  the 
Naval  Air  Systems  Command.  Points  noted  in  four  case  studies  were  contrasted  with 
policies  and  practices.  Three  essential  characteristics  of  the  multi-phase  source  selection 
strategy  are  that  phases  are  used  to  fully  understand  requirements  to  the  point  that 
program  risk  is  reduced,  changes  to  requirements  do  hot  have  to  b'd  re-competed  and  ho 
Justification  and  Approval  is  required  for  other  than  foil  and  open  competition  when 
going  into  a  follow-on  phase  considering  only  offerors  from  prior  phases.  Factors 
identified  when  the  proposed  multi-phase  source  selection  strategy  is  appropriate  include: 
(1)  confidence  in  the  ability  to  determine  a  fair  price  without  supplier  cost  data;  (2)  a 
fluid  requirement;  (3)  potential  to  take  advantage  of  commercial  technology;  (4)  time  to 
folly  accommodate  exploratory  phases  prior  to  or  at  the  start  of  the  program;  (5)  adequate 
commitment  of  funds  to  accommodate  potential  growth  in  funding  requirements  during 
requirement  development  phases;  (6)  experienced  personnel  willing  to  adopt  new 
strategies  and  engage  in  revised  behavior  patterns  and  (7)  organizational  willingness  to 
modify  regulatory  guidance  as  needed  to  accommodate  the  intended  strategy. 
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I. 


INTRODUCTION 


A.  BACKGROUND 

1.  Price-Based  Acquisition 

The  Department  of  Defense  (DoD)  looks  to  increased  use  of  Price-Based  Acquisition 
(PBA)  as  a' way  to  conduct  business  in  a  manner  that  is  more  closely  aliped  with  current 
practices  in  the  commercial  sector.  Through  increased  use  of  PBA  and  commercial  buying 
practices  three  objectives  are  sought:  (Ref  8) 

•  Increase  access  to  leading  edge  technology  available  only  in  the  commercial  sector 

•  Reduce  Government  infrastructure 

•  Obtain  better  value  in  what  is  purchased 

The  DoD  defines  PBA  as  a  way  to  buy  goods  and  services  that  does  not  rely  primarily  on 
a  supplier  providing  cost  data  (Ref  8). 

The  DoD  chartered  a  study  group  on  implementing  PBA.  This  group  authored  an 
extensive  report  (Ref  8)  in  which  a  number  of  conclusions  and  recommendations  were  made. 

The  DoD  had  used  fixed-price  development  contracts  in  the  1980s,  with  poor  results.  The  study 
group  noted  that  a  significant  issue  driving  those  poor  results  was  inadequate  communication 
between  industry  and  Government.  Communications  between  Government  and  industry  took 
place  via  a  formal  process.  This  formal  process  hindered  complete  understanding  of  complex 
technical  requirements  and  development  work.  The  study  group  met  with  industry 
representatives  to  determine  why  this  was  the  case.  Their  conclusion  was  that  the  commercial 
approach  of  collaborative  interaction  with  suppliers  led  to  an  increased  imderstanding  of  buyer 
requirements.  This  increased  level  of  understanding  facilitated  the  use  of  fixed-price  contracts. 
In  comparison  to  commercial  approaches,  statutory  and  regulatory  restrictions  in  Government 
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acquisitions  prevented  the  Government  from  achieving  this  increased  level  of  understanding 
between  buyer  and  seller.  (Ref  8) 

One  proposal  of  the  PBA  chartered  study  is  a  multi-phase  source  selection  process.  This 
process  is  intended  to  address  the  risk  present  in  acquisitions  due  to  constrained  understanding  of 
Government  requirements.  Facilitating  freer  and  more  open  communication  between  buyer  and 
seller  can  lower  the  risk  level  such  that  fixed-price  contracts  may  be  appropriate  in  complex 
acquisitions.  (Ref  8) 

The  DoD  undertakes  complex  acquisitions  and  procuring  organizations  are  risk  averse. 
This  combination  results  in  the  use  of  cost-based  contracts.  When  assessing  risk  at  the  program 
level  without  a  complete  imderstanding  of  requirements  by  both  industry  and  Government,  cost- 
based  contracts  generally  result.  Contracts  are  awarded  and  work  commences  before  a  complete 
understanding  of  the  requirements  is  achieved.  In  the  proposed  multi-phase  source  selection, 
risk  is  broken  down  into  discrete  increments  as  opposed  to  the  traditional  method  of  dealing  with 
risk  at  the  level  of  the  entire  program.  (Ref  8) 

The  recent  Federal  Acquisition  Regulation  (FAR)  Part  15  rewrite  advocated  using  a  more 
extensive  negotiating  process  with  the  highest  rated  offerors.  The  Government  uses  methods 
such  as  pre-solicitation  notices  or  Request  for  Proposals  (RFP)  to  communicate  requirements  to 
industry.  Acquisition  practitioners  recognize  shortcomings  in  the  execution  of  these  vehicles. 
FAR  Part  15  specifically  encourages  early  exchanges  of  information  about  future  acquisitions 
with  industry.  Feedback  to  the  Government  is  desired,  but  often  what  is  received  is  superficial. 
Industry  avoids  critical  feedback  of  requirements  to  avoid  offending  the  customer  and  providing 
insight  to  competitors  about  their  proposed  solutions.  (Ref  8). 

To  accurately  respond  to  a  solicitation,  offerors  need  to  make  trades  in  cost,  schedule  and 
performance.  In  order  to  make  these  trades,  alternative  proposals  must  be  submitted.  This 
presents  a  large  risk  to  the  offeror.  An  alternative  proposal  may  be  rejected  for  not  meeting  all 
requirements,  even  when  there  are  no  other  proposals  technologically  feasible  at  their  stated 
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prices.  Disclosure  gives  competitors  an  opportunity  to  mirror  trades  made  in  cost,  schedule  or 
performance.  An  outcome  of  this  process  is  overly  optimistic  technical  proposals.  Better 
alternatives  are  offeror  identification  of  unrealistic  requirements  or  more  realistic  price  estimates. 
Lack  of  requirement  understanding  results  in  the  use  of  cost-based  contracts.  The  Government 
ends  up  not  recognizing  the  need  to  alter  prohibitively  expensive  or  technologically  unachievable 
requirements.  This  is  a  significant  shortcoming  in  FAR  Part  15  methods.  (Ref  8) 

Risk-averse  acquisition  organizations  tend  to  include  all  offerors  in  the  processreven  •  » - 
when  allowed  to  eliminate  those  less  competitive  by  the  FAR.  The  acquisition  process  is 
fi-equently  pushed  forward  by  external  time  constraints,  such  as  fimding  timelines.  This  places 
limits  on  the  level  of  communication  between  industry  and  Government,  with  adverse 
imintended  consequences  surfacing  later  during  contract  execution.  (Ref  8) 

A  key  distinguishing  element  of  the  multi-phase  source  selection  strategy  is  that 
discussions  are  unique  to  each  offeror.  Treating  offerors  fairly  does  not  mean  that  the  same 
information  is  shared  with  each.  (Ref  8) 

In  the  proposed  multi-phase  source  selection,  sources  are  initially  selected  following 
evaluation  of  capability  statements.  The  capability  statement  includes  limited  information  such 
as:  (Ref  8) 


•  Past  performance  of  the  contractor  for  the  same  or  similar  work 

•  Ability  to  perform  the  required  work 

•  Conceptual  approach  to  performing  the  work 

•  Rough  order  of  magnitude  price  estimate  for  the  conceptual  approach 

Government  requirements  are  initially  stated  as  objectives,  and  may  be  refined  through 
one  or  a  succession  of  phases.  The  goal  is  to  reach  a  point  of  understanding  where  fixed-price 
contracts  can  be  used  to  allocate  risk  between  the  Government  and  industry.  The  number  of 
sources  competing  may  be  reduced  in  each  phase.  The  first  two  phases  consist  of  proposal 
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development  and  requirements  development.  Both  phases  are  not  necessary.  Sources  can  be 
selected  to  participate  in  follow-on  phases  based  on  being  included  in  an  earlier  phase.  Proposed 
statutory  language  allows  for  award  to  limited  sources  without  reopening  the  competition.  This 
change  negates  the  need  for  a  Justification  and  Approval  (J&A)  for  other  than  full  and  open 
competition  when  limiting  sources  in  this  manner.  (Ref  8) 

Government  requirements  can  continue  to  evolve  throughout  all  phases.  If  the 
Government  desires  to  do  this,  it  is  stated  upfront  in  the  solicitation.  Changes  can  be  made 
without  amending  the  solicitation.  There  is  no  stated  upper  limit  on  the  number  of  phases  used 
and  award  can  be  made  to  one  or  more  offerors.  (Ref  8) 

2.  Current  Practices 

The  US  Coast  Guard  is  using  an  innovative  evolutionary  source  selection  process  to 
award  a  contract  intended  to  replace  a  large  portion  of  its  assets.  The  assets  in  question  support 
missions  that  take  place  greater  than  fifty  miles  from  land.  This  is  called  the  Deepwater  mission. 
Timely  acquisition  of  resources  that  use  available  technology  is  a  stated  objective.  Commercial 
and  military  technology  combinations  are  being  innovatively  incorporated  in  a  system  of  systems 
deliverable. 

The  Deepwater  project  is  currently  at  the  point  of  completing  the  second  portion  of  phase 
I  activities,  that  of  functional  design.  Phase  I  consists  of  two  parts,  conceptual  design  and 
functional  design.  In  conceptual  design  the  contractors  developed  proposed  concepts  to  about 
the  50  percent  mark.  Three  out  of  four  competitors  were  awarded  phase  I  contracts  in  August 
1998.  The  prime  contractors  were  Lockheed  Martin,  Avon-Dale  Industries,  Inc.,  and  Science 
Applications  International  Corporation  (SAIC).  In  the  functional  design  phase  the  industry 
teams  refined  their  concepts  to  about  the  80  percent  mark.  The  start  of  phase  II  is  a  competitive 
decision  point.  The  Coast  Guard  intends  to  award  a  contract  for  detailed  design  and  construction 
to  one  of  the  three  industry  teams  in  January  2002.  This  award  will  be  based  on  an  evaluation  of 
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proposals  in  terms  of  maximizing  operational  effectiveness  while  minimizing  total  ownership 
cost.  (Ref  17) 

The  Deepwater  source  selection  methodology  was  developed  independently  from  the 
refinement  of  the  multi-phased  source  selection  methodology  outlined  in  the  DoD  PBA  report. 
Issues  surfaced  in  the  planning  and  execution  of  the  Deepwater  source  selection  are  both  similar 
and  relevant  to  issues  addressed  in  the  proposed  multi-phase  source  selection  methodology.  The 
Deepwater  project  represents  one  of  a  numberof  Federal  procurements  from  which  lessons  can 
be  learned  applicable  to  the  implementation  of  a  multi-phase  source  selection  process. 

3.  Current  Policy 

At  the  time  of  this  writing,  the  Office  of  Federal  Procurement  Policy  and  the  DoD 
Acquisition  Reform  Office  have  completed  an  initial  evaluation  of  PBA  study  team 
recommendations,  including  proposed  statutory  changes  to  facilitate  implementation  of  the 
multi-phase  source  selection  strategy.  They  have  issued  a  call  for  Services  to  submit  pilot 
projects  to  implement  report  recommendations.  The  Naval  Air  Systems  Command  (NAVAIR)  is 
considering  a  pilot  project  in  which  to  utilize  and  fiirther  evaluate  this  methodology. 

B.  PURPOSE 

This  thesis  will  take  an  in-depth  look  at  points  noted  by  major  system  acquisition 
practitioners  in  four  case  studies.  The  points  noted  will  be  contrasted  with  current  source 
selection  practices  and  policies  at  NAVAIR.  Conclusions  will  be  drawn  regarding  the 
implementation  of  the  proposed  multi-phased  source  selection  strategy  in  support  of  PBA, 
specifically  at  NAVAIR. 

This  study  will  further  advance  the  implementation  of  PBA  in  the  DoD,  specifically  at 
NAVAIR.  This  study  will  also  assist  the  DoD  Acquisition  Reform  Office  in  evaluating  PBA 
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report  recommendations,  specifically  in  evaluating  proposed  statutory  changes  with  the 
Congress. 

C.  RESEARCH  QUESTIONS 

Primary:  What  are  the  essential  characteristics  of  a  NAVAIR  source  selection  process 
that  incorporates  to  the  maximum  extent  possible  multi-phase  source  selection  as  recommended 
in  the  1999  PBA  report? 

Secondary: 

1 .  What  is  a  multi-phase  source  selection  strategy,  and  how  does  it  differ  from  present 
source  selection  strategies? 

2.  How  does  the  multi-phase  source  selection  strategy  recommended  in  the  PBA  report  of 
December  1999  differ  from  multi-phase  strategies  currently  used  by  NAVAIR? 

3.  To  what  extent  does  use  of  a  multi-phase  source  selection  strategy  facilitate  PBA? 

4.  What  are  the  potential  benefits  and  risks  of  utilizing  a  multi-phase  source  selection 
strategy? 

5.  What  conditions  best  support  the  use  of  a  multi-phase  source  selection  strategy? 

6.  How  have  other  Federal  procurements  overcome  barriers  and  utilized  a  multi-phase 
source  selection  strategy? 

7.  What  barriers  exist  that  preclude  NAVAIR  from  utilizing  a  multi-phase  source  selection 
strategy? 

8.  What  should  the  underlying  NAVAIR  procedure  be  to  fully  incorporate  multi-phase 
source  selection? 

9.  What  are  the  pertinent  factors  for  screening  potential  programs  for  utilization  of  a  multi¬ 
phase  source  selection  strategy  at  NAVAIR? 
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D.  SCOPE  AND  METHODOLOGY 


1.  Scope 

The  scope  of  this  thesis  is  limited  to  extrapolation  of  observations  made  in  four  case 
studies.  Assumptions  are  that  the  reader  is  knowledgeable  of  Government  acquisition  terms  and 
procedures.  A  literature  review  of  source  selection  methodology  will  be  conducted  and 
presented  to  provide  background  on  source  selection  strategies^  with- features  similar  to  those  of- 
the  proposed  multi-phase  strategy.  A  review  of  recent  literatmre  and  discussions  with  program 
managers  and  support  personnel  will  be  used  to  select  four  case  studies  that  highlight  innovations 
achieved  in  source  selection.  The  scope  of  the  study  is  focused  on  potential  use  of  the  proposed 
multi-phase  source  selection  strategy  at  NAVAIR. 

2.  Methodology 

The  methodology  used  in  this  thesis  research  consists  of  the  following  steps: 

1.  Conduct  a  literature  search  of  books,  magazine  articles,  CD-ROM  systems,  Internet  and 
other  library  information  resources. 

2.  Conduct  interviews  with  source  selection  practitioners  and  policy  makers  in  relevant 
Federal  programs,  at  NAVAIR,  the  Office  of  Federal  Procurement  Policy  and  at  the  DoD 
Acquisition  Reform  Office. 

3.  Interpret  points  noted  by  source  selection  practitioners  in  four  case  studies  using  multi¬ 
phase  or  similar  source  selection  processes  in  Federal  procurements 

4.  Apply  the  interpretation  of  points  noted  in  the  case  studies  to  evaluate  the  potential  for 
utilization  of  a  multi-phase  source  selection  strategy  at  NAVAIR,  specifically  in  a  major 
system  acquisition. 
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E.  ORGANIZATION  OF  STUDY 


This  thesis  is  organized  into  five  Chapters.  Chapter  I  provided  an  introduction  to  the 
subject  of  study  and  an  overview  of  research  questions,  scope  and  methodology.  Chapter  II 
presents  source  selection  methodologies  and  their  intended  objectives.  The  source  selection 
strategies  presented  in  Chapter  II  do  not  represent  all  possible  source  selection  strategies.  The 
source  selection  strategies  presented  represent  those  with  objectives  and  methods  similar  in 
nature  to  the  proposed  multi-phase  source  selection  strategy.  ■ 

Chapter  III  describes  four  acquisition  program  case  studies.  This  Chapter  presents  an 
objective  look  at  points  noted  by  program  managers  and  support  personnel  during  the  execution 
of  source  selection.  Expected  benefits  from  use  of  the  strategies  employed,  as  well  as  barriers, 
challenges  and  advantages  are  highlighted. 

Chapter  IV  presents  an  overview  of  source  selection  policies,  practices  and  procedures  at 
NAVAIR.  Organizational  efforts  to  improve  source  selection  at  NAVAIR  are  also  presented. 

Chapter  V  presents  interpretations  of  the  points  noted  in  the  Chapter  III  case  studies. 
These  interpretations  are  then  contrasted  with  the  source  selection  policies  and  practices  at 
NAVAIR  and  with  the  proposed  multi-phase  source  selection  strategy.  This  contrasting  is 
utilized  to  draw  conclusions  regarding  the  use  of  the  proposed  multi-phase  source  selection 
methodology  at  NAVAIR,  and  in  support  of  PBA. 

Two  appendices  are  provided  after  Chapter  V.  The  first  presents  factors  for  screening 
potential  programs  for  utilization  of  a  multi-phase  source  selection  strategy  at  NAVAIR.  The 
second  provides  language  to  incorporate  the  multi-phase  source  selection  strategy  into  NAVAIR 
source  selection  guidance. 
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II.  OVERVIEW  OF  SOURCE  SELECTION  METHODOLOGY 


The  purpose  of  this  Chapter  is  to  present  background  information  on  acquisition/source 
selection  strategies  similar  in  nature  to  multi-phase  source  selection  and  to  introduce  the  multi¬ 
phase  source  selection  strategy.  This  Chapter  provides  a  backdrop  from  which  to  analyze  the  use 
of  a  multi-phase  source  selection  strategy. 

Source  selection  strategy  is  embedded  in  acquisition' strategy^  An  acquisition  strategy  is  a 
big  picture  outline  of  how  program  objectives  are  to  be  achieved  with  the  level  of  resources 
provided  (Ref  4).  This  high-level  strategy  includes  defining,  developing  and  fielding  a  weapon 
system.  A  source  selection  methodology  achieves  one  part  of  an  acquisition  strategy,  to  select 
the  appropriate  offeror  to  award  a  contract.  During  execution  of  an  acquisition  strategy,  more 
than  one  contract  may  be  awarded.  The  intertwinement  of  acquisition  strategies  and  source 
selection  methodologies  requires  discussion  of  both  to  fully  understand  the  context  in  which  a 
source  selection  will  take  place. 

A.  BACKGROUND 

1.  Source  Selection  in  General 

Source  selection  includes  a  solicitation  for  and  evaluation  of  proposals.  A  source  is 
selected,  a  contract  is  awarded  and  unsuccessful  offerors  are  debriefed.  Source  selection  is 
governed  by  statute  and  regulation.  For  Navy  procurements  this  includes  the  Federal  Acquisition 
Regulation  (FAR),  Defense  FAR  Supplement  (DFARS),  and  the  Navy  Acquisition  Procedures 
Supplement  (NAPS)  (Ref  11).  DoD  Directive  5000. 1 ,  DoD  Regulation  5000.2-R  and 
Comptroller  General  decisions  also  influence  the  process. 

Although  numerous  policies  and  procedures  must  be  adhered  to,  an  underlying  theme  is 
that  source  selection  practitioners  are  expected  to  exercise  sound  business  judgment  in  the 
application  of  rules  and  procedures.  Each  acquisition  stands  alone  and  the  source  selection 
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process  can  be  tailored  to  each  acquisition.  Legal  precedent  is  that  the  source  selection  decision 
must  result  from  rational  rather  than  arbitrary  judgment.  (Ref  11) 

Objectives  of  source  selection  include:  (Ref  11) 

•  Selecting  the  offeror  with  the  proposal  that  is  in  the  best  interest  of  the  Government. 

•  Ensuring  impartial,  equitable,  and  comprehensive  evaluation  of  proposals  and 
capabilities 

•  Minimizing  process  cost  to  both  Government  and  industry 

•  Documenting  the  basis  for  the  selection  decision 

Traditional  source  selection  is  used  when  factors  other  than  cost  or  price  are  considered 
prior  to  contract  award.  A  major  system  development  may  include  more  than  one  contract. 
Separate  contracts  may  be  awarded  for  development,  production,  fielding,  deployment,  operation 
and  support.  (Ref  11) 

Source  selection  generally  follows  this  sequence  of  events:  (Ref  11) 

Step  Action 

1  Source  Selection  Authority  (SSA)  designated 

2  Evaluation  groups  established 

3  Source  Selection  Plan  (SSP)  developed 

4  SSP  approved 

5  RFP  developed  and  issued 

6  Proposals  received  and  evaluated 

7  If  necessary,  discussions  are  held 

8  Final  proposal  revisions  received  and  evaluated 

9  Supporting  documentation  for  selection  decision  prepared 

10  Source  selected 

1 1  Chain  of  acquisition  authority  briefed  as  applicable 
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12  Contract(s)  awarded 

13  Unsuccessful  offerors  debriefed 

14  Lessons  learned  documented 

The  next  section  describes  two  phase  acquisition. 

2.  Two  Phase 

Two  phase  acquisitions  start  with  offerors  providing  past  performance  information,  a 
conceptual  outline  of  their  technical  approach  and  rough  order  of  magnitude  pricing.  The 
Government  evaluates  this  information  and  advises  offerors  whether  or  not  they  are  competitive 
for  further  consideration.  The  idea  is  to  reduce  the  number  of  detailed  proposals  received  in  the 
second  phase.  This  reduction  in  proposals  saves  both  the  Government  and  industry  resources. 
This  saving  of  resources  is  intended  to  attract  firms  that  historically  did  not  do  business  with  the 
Government  because  of  excessive  efforts  required  to  prepare  proposals  that  frequently  do  not 
result  in  contract  award.  (Ref  2) 

The  second  phase  includes  the  development  and  evaluation  of  detailed  proposals.  Only 
offerors  with  a  solid  chance  of  winning  the  contract  award  should  complete  detailed  proposals. 
However,  it  frequently  occurs  that  phase  two  solicitations  are  issued  to  all  first  phase  offerors  as 
well  as  to  offerors  that  did  not  participate  in  the  first  phase.  This  serves  to  defeat  the  purpose  of 
the  two  phase  strategy. 

The  Office  of  Federal  Procurement  Policy  Act  and  the  Small  Business  Act  require  that  all 
responsible  sources  have  the  opportunity  to  submit  offers.  Some  agencies,  like  the  Federal 
Aviation  Administration  (FAA)  are  exempt  from  this  requirement  and  have  the  ability  to  limit 
selection  of  second  phase  offerors  to  those  that  participated  in  the  first  phase  (Ref  32).  Limiting 
the  number  of  offerors  in  this  manner  enables  both  Government  and  industry  to  sharpen  their 
focus.  This  sharpened  focus  can  result  in  finding  an  optimal  match  between  Government 
requirements  and  solutions  available  in  the  market.  (Ref  2) 
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Risk  is  mitigated  in  the  two  phase  strategy  by  increasing  communication  between 
industry  and  Government.  Offerors  are  more  likely  commit  to  a  full  scale  phase  two  effort  to 
understand  the  requirement  and  explore  solutions.  The  risk  of  loss  of  proposal  development  cost 
is  lessened  if  an  offeror  knows  that  it  is  competing  in  a  limited  pool  in  which  it  stands  a  fair 
chance  of  receiving  the  contract  award.  (Ref  2) 

Competition  is  enhanced  with  freer  communication  between  prospective  offerors  and  the 
Government.  Requirement  and  evaluation  criteria  development  is  enhanced.  The  Government 
receives  the  most  benefit  from  competition  with  best  value  proposals  that  meet  all  requirements. 

It  is  better  to  end  up  evaluating  a  small  number  of  good  proposals  than  a  larger  number  of 
proposals,  some  of  which  do  not  meet  requirements  or  do  not  represent  good  value.  (Ref  2) 

Either  cost-based  or  price-based  acquisition  can  be  used  appropriately  with  a  two  phase 
strategy.  Acquisition  type,  such  as  design  or  production,  typically  determines  which  is 
appropriate.  Risk  exposure  may  be  limited  in  development  work  with  cost-beneficial  ceilings  set 
to  indicate  when  to  reevaluate  continuing  the  project.  (Ref  2) 

The  next  section  describes  phased  down  select  acquisition. 

3.  Phased  Down  Select 

“Down  select”  generally  means  to  reduce  the  number  of  contractors  working  on  a 
program  by  eliminating  one  or  more  for  the  next  phase. 

Performance  based  requirements  are  outlined  vice  using  design  specifications.  Offerors 
are  not  limited  to  one  solution  or  approach.  Offerors  can  be  evaluated  against  this  broader 
statement  of  requirements.  Offeror  submissions  may  progress  from  limited  outlines  in  early 
phases  to  detailed  or  full  proposals  in  follow  on  phases.  Full  proposals  may  not  be  solicited  from 
less  competitive  sources  or  a  cut  off  date  for  submission  of  full  proposals  may  be  used.  (Ref  7) 
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Phased  competition  through  down  selecting  is  intended  to  reduce  risk.  Risk  exposure  is 
limited  by  the  scope  of  each  phase.  Risk  is  limited  in  less  comprehensive  contracts  to  develop 
the  system  and  the  processes  to  be  used. 

Leading  edge  technology  solutions  requiring  significant  development  fit  well  with  phased 
competition.  The  exploration  of  commercially  available  solutions  may  indicate  that  limited  or 
significant  development  work  is  required  to  satisfy  the  requirement.  Meetings  with  prospective 
offerors,  pre-solicitation  notices  and  conferences,  draft  requests  for  proposals;  pre-proposal 
conferences  and  performance  specifications  and  statements  of  work  can  be  used  to  engage 
potential  offerors  in  the  search  for  solutions.  Risk  exposure  increases  when  a  previously  unused 
commercial  solution  or  a  solely-Govemment  developed  solution  is  used.  The  industry  risk  is 
that  of  preparing  a  proposal  without  winning  the  contract  award.  The  Government  risk  is  that  the 
solution  will  not  meet  requirements  or  will  not  be  the  best  solution.  Limiting  the  scope  of  the 
development  work  by  using  phases  can  reduce  these  risks.  (Ref  7) 

Two  phase  is  considered  by  some  practitioners  to  be  an  example  of  a  phased  down  select. 

The  next  section  describes  competitive  prototyping. 

4.  Competitive  Prototyping 

Competitive  prototyping  consists  of  the  development  of  prototypes  by  offerors.  Different 
designs  are  compared  in  prototype  form.  Offerors  can  complete  some  of  the  development  of  the 
design  in  making  the  prototype.  The  desired  outcome  is  that  the  completion  of  the  prototype  and 
the  development  work  are  completed  together.  The  Government  then  can  evaluate  a  proptotype 
that  is  ready  for  full-scale  production.  (Ref  2) 

Prototype  development  as  a  part  of  system  development  enhances  commtmication 
between  the  Government  and  industry.  End-users  can  see  and  operate  the  proposed  solution. 
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Marketplace  capabilities  can  be  demonstrated,  including  innovative  or  alternate  solutions  to 
Government  requirements. 

Risk  is  reduced  through  this  increased  level  of  communication.  Proving  that  designs  will 
work  as  intended  prior  to  committing  to  large-scale  efforts  and  associated  funding  also  reduces 
risk.  In  addition,  the  Government  gains  insight  into  the  cost  of  operational  support,  and  fit 
within  the  intended  operational  environment.  Risk  may  be  mitigated  to  the  point  that  fixed-price 
contracts  are  appropriate  earlier  in  system  development.  (Ref  2)  • 

A  prototype  can  be  anywhere  on  the  spectrum  from  a  complete  system  to  a  major 
component  to  a  partially  complete  model.  The  level  of  funding  committed  by  the  Government  to 
each  developer  may  differ.  Factors  to  consider  when  considering  funding  include  how  much 
funding  the  contractor  is  providing,  the  complexity  of  the  design  and  commercial  applicability  of 
the  prototype.  Commercial  applicability  of  the  output  increases  offeror  willingness  to  invest  in 
the  prototype  (Ref  2). 

Fixed-price  contracts  for  prototypes  can  fit  when  a  relatively  small  amount  of 
development  takes  place  as  the  prototype  is  built.  This  limits  the  development  risk.  Allocating 
risk  in  this  manner  is  most  likely  to  be  the  result  of  further  development  of  a  commercial  or  non- 
developmental  item.  (Ref  2) 

Competition  can  be  maintained  in  two  ways:  awarding  two  or  more  prototype  or 
combined  prototyping  and  development  contracts  and  upgrading  the  current  system  with  limited 
development. 

The  next  section  describes  evolutionary  acquisition. 
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5. 


Evolutionary  Acquisition 


In  evolutionary  acquisition,  a  system  with  a  well-defined  baseline  capability  is  fielded 
early  on.  Enhancements  to  this  baseline  capability  are  fielded  imtil  the  system  is  complete.  (Ref 
3)  An  overview  of  this  approach  is  shown  in  Figure  2- 1 . 

The  source  selection  and  acquisition  strategies  presented  so  far  wrestle  with  successfully 
navigating  shifting  requirements  and  priorities.  Evolutionary  acquisition  embodies  a  numberof 
common  threads  that  run  through  these  strategies.  The  goal  is  to  successfully  field  a  system 
within  a  framework  of  emerging  technologies  and  budgetary  pressures. 


Level  of 
User  Insight 


standard  Approach 

Insight  at  Beginning 
With  Large  Gap  In 
Between 


Level  of 
User  Insight 


Evolutionary  Approach 

Early  and  Continuous  User 
Interaction  and  Technology  Input 


rr _ 


Doesn’t  Work  for 

•  Provides  Mechanism  to  Solicit  User  and  Technology 

Unprecedented 

Feedback  and  Make  Required  Adjustments 

Systems  or 

But, 

Rapidly  Changing 

•  Uncertainty  Makes  it  Difficult  to  Accurately  Quantify 

Environments 

Cost,  Schedule  for  this  Activity  in  "Advance” 

Figure  2-1.  Evolutionary  Acquisition  Overview  (From  Ref  3) 
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Each  increment  of  capability  acquired  is  treated  as  an  individual  acquisition.  Defining 
the  boundaries  of  the  increments  requires  feedback  from  the  end  users,  developers,  testers  and 
maintainers.  The  insertion  of  developing  technology  within  the  boundaries  of  the  schedule,  the 
requirement  and  the  cost  has  to  be  considered  as  well  as  the  associated  risks.  Increased 
communication  is  required  between  the  participants  listed  (Ref  3).  Heavy  involvement  is 
required  from  Plarming,  Programming  and  Budgeting  System  participants  to  ensure  that  funding 
timelines  can  support  the  discontinuous  funding  profile  that  emerges  when  using  an  evolutionary 
strategy 

Characteristics  of  evolutionary  acquisition  are:  (Ref  3) 

•  A  general  description  of  the  functional  capability  desired  for  the  full  system.  The 
desired  capabilities  of  the  full  system  are  not  specified  in  detail 

•  A  concise  statement  of  operational  concepts  for  the  full  system 

•  A  flexible,  well  planned  overall  architecture  incorporating  a  process  to  include 
changes 

•  A  plan  for  incrementally  achieving  the  desired  total  capability  which  adheres  to  life 
cycle  cost  effectiveness 

•  Early  definition,  funding,  development,  testing,  fielding,  supporting  and  operational 
evaluation  of  an  initial  increment  of  operational  capability 

•  Continual  dialogue  and  feedback  among  users,  developers,  supporters  and  testers 

Successful  execution  requires  identification  of  and  adherence  to  a  system  architecture 
that  can  accommodate  additional  performance  modifications. 

The  next  section  describes  modular  contracting. 
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6. 


Modular  Contracting 


In  modular  contracting  an  acquisition  is  separated  into  stand-alone  modules.  Delivery, 
implementation,  and  testing  of  the  entire  system  is  accomplished  in  increments  (Ref  1).  Each 
increment  stands  alone,  and  is  separately  and  fully  funded.  The  end  product  of  one  module  can 
be  built  upon  in  the  next  module.  Modules  can  be  accomplished  in  series  or  in  parallel.  If 
follow-on  modules  were  not  part  of  the  original  plan,  it  may  be  more  beneficial  to  award  them  to 
the  inciunbent  contractor  than  competing  the  follow-on  modules-  ' 

As  with  evolutionary  acquisition,  a  system  architecture  that  is  open  or  can  accommodate 
follow-on  modules  is  required.  The  greatest  risk  in  modular  contracting  is  integration  risk. 
Integration  can  be  the  responsibility  of  the  Government  or  a  contractor,  but  it  must  be  addressed. 

Risk  is  mitigated  in  several  ways.  Acquiring  a  system  via  smaller  modules  allows  the 
Government  to  make  fact-based  decisions.  Traditional  acquisition  methods  require  more 
reliance  on  projections  and  estimates.  By  breaking  down  a  larger  acquisition  into  smaller 
modules,  increased  access  to  industry  and  smaller  firms  is  achieved,  enhancing  competition. 
Government  flexibility  is  increased  through  an  increased  ability  to  accommodate  shifts  in 
technology,  and  an  increased  ability  to  start,  stop  or  change  program  direction  or  timelines  at  the 
completion  of  each  module.  Modular  contracting  is  results-based  as  opposed  to  process-based. 

It  can  enhance  program  management  stability  and  risk  assessment  under  conditions  of  rapidly 
changing  technology.  (Ref  1) 

Contracted  work  scope  is  smaller  and  more  likely  to  be  accomplished  within  goals  set  by 
the  contractor  when  compared  to  traditional  developmental  acquisition.  Acquisitions  can  also  be 
terminated  with  smaller  sunk  costs  if  it  becomes  apparent  that  threshold  goals  will  not  be  met. 

The  risk  mitigation  avenues  available  in  modular  contracting  may  facilitate  fixed-price 
contracts  where  not  achievable  in  a  large  developmental  program.  In  cost-based  contracting, 
modules  serve  to  limit  Government  financial  exposure.  (Ref  1) 
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Although  modular  contracting  is  generally  referred  to  in  the  acquisition  of  major 
Information  Technology  (IT)  systems,  it  can  be  used  in  the  development  of  other  types  of 
systems. 

The  next  section  describes  incremental  acquisition. 

7.  Incremental  Acquisition 

Incremental  acquisition  divides  the  work  effort  into  overlapping  phases  (Ref  8).  Phases 
can  be  fixed-priced.  Because  the  output  of  the  design  phase  is  needed  as  an  input  to  build  the 
solicitation  for  some  follow-on  increments  like  production,  some  fixed-price  phases  cannot 
overlap.  Figure  2-2  shows  incremental  acquisition  in  a  development  program. 


Figure  2-2.  Incremental  Development  Acquisition  Strategy  (From  Ref  8) 
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Acquisition  utilizing  an  incremental  strategy  is  essentially  the  same  as  that  described 
under  evolutionary  contracting,  although  there  is  some  disagreement  among  practitioners  about 
the  precise  definitions  of  modular,  evolutionary  and  incremental  contracting. 

The  next  section  describes  Other  Transaction  Authority. 

8.  Other  Transaction  Authority  (OTA) 

In  1994,  Congress  passed  Public  Law  103-160.  Section  845  of  this  law  authorized  the 
use  of  OTA  in  Federal  Acquisitions.  OTA  is  not  clearly  defined,  other  than  to  say  what  it  is  not. 
This  points  out  that  OTA  does  not  represent  doing  business  as  usual  through  contracts  or  grants. 

It  is  not  subject  to  the  laws,  rules  and  regulations  governing  traditional  defense  acquisitions. 

OTA  allows  iimovative  acquisition  strategies,  tailored  beyond  that  which  is  possible  in  the 
traditional  acquisition  process  (Ref  5).  Government  acquisitions  are  frequently  tailored  to  an 
acquisition  process.  OTA  enables  tailoring  of  the  contract  process  to  each  project,  rather  than 
the  reverse. 

OTA  has  successfully  been  used  to:  (Ref  5) 

•  Stimulate  access  to  firms  that  normally  do  not  do  business  with  the  DoD 

•  Integrate  commercial  and  military  processes 

•  Provide  flexibility  in  the  area  of  patent  and  data  rights  and  relief  from  flow-down 
procurement  clauses 

•  Facilitate  an  acquisition  approach  based  upon  a  multi-phase  down  select  of  competing 
contractors  (where  the  initial  phase  results  would  serve  as  the  valuation  criteria  for 
the  award  of  the  follow-on  phases) 

The  Defense  Advanced  Research  Program  Agency  (DARPA)  is  a  leader  in  the  use  of 
OTA.  DARPA  used  OTA  as  a  way  to  achieve  several  objectives.  These  objectives  include 
reducing  timelines  for  development,  reducing  costs,  and  shifting  the  focus  of  acquisition 
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practitioners  away  from  compliance  with  regulations.  OTA  allows  the  use  of  Generally 
Accepted  Accounting  Practices  in  place  of  the  generally  more  restrictive  Government  Cost 
Accounting  Standards.  OTA  does  require  adherence  to  applicable  fiscal  and  socio-economic 
laws.  Regulations  such  as  the  Armed  Services  Procurement  Act,  Competition  in  Contracting  Act 
(CICA),  FAR  and  DFARS  do  not  have  to  be  strictly  adhered  to.  (Ref  6) 

Combining  OTA  with  common  sense  and  non-rigid  interpretation  of  governing 
regulations  opens  the  door  for  innovation  in  processes  by  both  the  Government  and  industry. 
(Ref  5) 


OTA  is  intended  to:  (Ref  6) 

•  Increase  the  number  of  commercial  firms  participating  in  Government  projects 

•  Expand  opportunities  to  leverage  commercial  technologies 

•  Use  broad  performance  characteristics  to  identify  requirements 

•  Increase  the  level  of  design  responsibility  assigned  to  contractor  teams 

•  Use  smaller  Government  program  offices 

The  removal  of  regulatory  constraints  that  encompass  acquisitions  permits  use  of  a  wide 
array  of  source  selection  and  acquisition  strategies.  For  this  reason,  OTA  is  considered  to  be  a 
hybrid,  as  opposed  to  a  stand-alone  acquisition  or  source  selection  strategy. 

The  next  section  discusses  PBA  and  multi-phase  source  selection  as  an  element  of 
acquisition  reform. 

B.  PRICE-BASED  ACQUISITION  AND  MULTI-PHASE  SOURCE  SELECTION 

Over  the  last  20  years,  the  DoD  has  initiated  numerous  acquisition  reform  initiatives. 

The  FAR  part  15  rewrite  represents  one  of  these  reform  initiatives.  FAR  Part  15  covers  the  basic 
rules  for  negotiated  procurements.  The  most  controversial  aspect  of  the  FAR  Part  1 5  reAvrite  has 
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been  the  initiative  to  let  agencies  narrow  the  competitive  range.  What  was  “when  in  doubt  keep 
them  in”  became  “when  in  doubt  keep  them  out.”  Contracting  Officers  no  longer  had  to  include 
borderline  proposals.  Instead,  the  competitive  range  should  contain  only  “all  of  the  most  highly 
rated  proposals.”  This  standard  is  subject  to  two  exceptions.  First,  the  Contracting  Officer  can 
restrict  the  competitive  range  to  maintain  efficient  competition,  if  the  solicitation  states  this 
possibility  up  firont.  Second,  the  Contracting  Officer  can  eliminate  an  offeror  from  the 
competitive  range  during  discussions,  even  if  all  material  aspects  of  the  proposal  have  not  been 
discussed:'" 

This  rewrite  authorized  the  use  of  advisory  down  selects.  In  an  advisory  down  select,  the 
Government  can  ask  for  preliminary  information  and  use  this  information  to  discourage  less 
competitive  offerors.  However,  even  if  an  offeror  is  told  that  it  is  not  a  viable  competitor,  a  new 
proposal  can  still  be  submitted. 

The  rewrite  included  a  section  on  the  exchange  of  information  between  an  agency  and  a 
prospective  contractor.  The  rewritten  regulation  promotes  fi-eer  communication  between 
industry  and  Government.  The  terms  communications,  discussions  and  clarifications  were 
redefined.  Clarifications  were  expanded  from  corrections  of  minor  or  clerical  errors  to  include 
responding  to  past  performance  ratings.  Communications  occur  between  a  vendor  and  an  agency 
before  the  establishment  of  the  competitive  range  and  assist  in  that  process.  The  rewrite  defines 
discussions  as  negotiations  in  the  context  of  a  competitive  procurement.  These  can  now  include 
talks  about  contract  requirements,  such  as  clauses  that  are  not  mandatory.  The  revised  definition 
of  discussions  includes  the  Government  telling  offerors  to  expand  their  proposals  beyond  the 
minimum  requirements.  The  Government  can  also  suggest  that  an  offer  would  be  more 
competitive  if  it  had  fewer  extras  and  a  lower  price.  The  expansion  of  these  definitions  was 
intended  to  address  the  problem  of  vague  and  uninformative  evaluation  criteria. 

The  rewrite  promotes  freer  communication  between  industry  and  Government,  especially 
during  the  pre-proposal  stage.  However,  the  final  regulation  reflects  something  of  a  compromise 
and  two  common  criticisms  are  a  lack  of  clarity  and  that  the  intended  goals  are  not  materializing. 
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Coming  close  on  the  heels  of  the  FAR  Part  15  rewrite,  Section  912(c)  of  the  Fiscal  Year 
1998  Defense  Authorization  Act  directed  the  Secretary  of  Defense  to  submit  an  implementation 
plan  to  Congress  to  streamline  acquisition  organizations,  workforce,  and  infrastructure.  To 
achieve  the  DoD  vision  laid  out  by  the  Secretary  of  Defense  in  April  1998,  groups  were 
chartered  to  study  the  initiatives  that  comprise  the  vision.  The  individual  initiatives  included: 

•  Research,  Development  Technology  and  Engineering  (RDT&E)  infrastructure 

•  C3  integration/acquisition 

•  Product  support 

•  Program  Management  (PM)  life-cycle  management 

•  Training  for  service  contracting 

•  Continuous  learning 

•  Training  for  commercial  business  environment 

•  Technical  workforce  requirement  and  retention 

•  Future  acquisition  and  technology  workforce 

•  Paperless  contracting 

•  Paperless  integrated  data  environment 

•  PBA 

•  Integrated  test  and  evaluation 

•  Requirements/acquisition 

The  912(c)  PBA  Team  was  chartered  to:  Analyze  the  implementation  of  PBA  in  the 
DoD,  identify  specific  tools  and  techniques  to  facilitate  greater  use  of  PBA  within  the 
Department,  and  to  identify  what  actions  (statutory,  regulatory,  and  policy)  will  be  required  to 
transform  the  Department’s  buying  practices  into  ones  that  are  more  commercial-like.  (Ref  8) 

This  team  defined  PBA  as  a  way  for  DoD  to  buy  goods  and  services  that  does  not  rely 
primarily  on  a  supplier  providing  cost  data,  stating  that  PBA  is  a  way  of  doing  business  that 
begins  with  identification  of  a  need  and  flows  through  post-award  activities.  The  study  group 
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saw  PBA  as  a  way  to  become  more  commercial  in  buying  practices  and  achieve  three  outcomes: 
(1)  increase  access  to  leading  edge  technology  available  in  the  commercial  sector,  (2)  reduce 
Government  infrastructure  and  (3)  get  better  prices  for  what  the  DoD  buys  (Ref  8). 

Understanding  the  application  of  PBA  is  facilitated  through  the  use  of  models.  In  the 
model  shown  in  Figure  2.3,  a  price-based  approach  is  associated  with  a  high  level  of 
competition,  high  confidence  in  the  ability  to  use  price  analysis  to  determine  a  fair  and 
reasonable  price,  and  low  technical  risk.  (Ref  8)  ■  ■ 


Figure  2.3.  CBA  to  PBA  Continuum  I  (From  Ref  8) 

A  cost-based  approach  is  shown  in  this  model  coinciding  with  low  competition, 
confidence  in  the  ability  to  use  price  analysis  to  determine  a  fair  and  reasonable  price,  and  low 
technical  risk. 

The  two  ends  of  the  model  are  not  mutually  exclusive.  A  combination  acquisition 
strategy  can  be  used.  The  peculiarities  of  any  given  acquisition  drive  the  resultant  choice  of 
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strategy.  The  three-dimensional  focus  of  Figure  2.3  leaves  out  the  purpose  of  the  acquisition  and 
equally  weights  the  three  factors.  It  also  does  not  depict  pre-  and  post-award  activities.  The 
PBA  team  referred  to  pre-  and  post-award  activities  as  the  business  case  development  of  an 
acquisition.  (Ref  8) 

An  alternate  depiction  of  the  cost  to  price-based  continuum  is  showm  in  Figure  2.4.  In 
this  depiction,  a  series  of  continuum  are  presented  in  three  categories;  acquisition  strategy, 
enablers  'and  business  case  development.  The  authors  of  this  depiction  stressed  that  the  three  • ' 
groupings  of  continuum  are  not  necessarily  of  equal  importance. 

In  Figure  2.4,  acquisition  strategy  is  placed  on  top  to  emphasize  the  planning  process.  An 
acquisition  for  research  and  development  has  different  inherent  risks  associated  with  it  than  does 
an  acquisition  for  production.  Frequently,  consideration  for  mitigation  of  this  inherent  risk 
drives  the  decision  to  use  either  a  cost  or  a  price-based  approach.  (Ref  8) 


Figure  2.4,  CBA  to  PBA  Continuum  II  (From  Ref  8) 
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The  enabler  group  shares  two  sub-elements  with  Figure  2.3,  degree  of  competition  and 
confidence  in  the  ability  to  obtain  a  fair  and  reasonable  price  without  supplier  cost  data.  The 
point  of  the  enabler  subsection  is  that  market  conditions  should  not  solely  determine  whether  a 
price  or  cost-based  approach  is  more  appropriate.  As  with  all  of  the  sub-elements  in  this  model, 
the  degree  of  competition  is  not  by  itself  a  prerequisite  for  PBA.  Instead,  competition  facilitates 
or  enables  PBA.  PBA  may  be  used  without  competition,  as  in  a  sole-source  market.  (Ref  8) 

The  bottom  grouping  in  Figure  2.4  is  labeled  business  case  development.  Addressed 
within  this  set  of  continuum  is  contract  type,  use  of  cost  data  and  post-award  methodology.  Each 
sub-element  contributes  to  the  level  of  support  given  for  either  cost-  or  price-based  strategies.  If 
the  characteristics  of  all  continua  shown  weigh  more  heavily  to  the  left  of  Figure  2.4,  then  cost- 
plus  contracts  with  the  incentive  on  cost  are  most  appropriate.  If  the  opposite  is  true,  firm-fixed- 
price  (FFP)  and  fixed-price  with  performance  incentive  contracts  are  more  appropriate.  (Ref  8) 

Taken  as  a  whole,  it  is  difficult  to  label  an  acquisition  as  either  cost  or  price-based, 
because  many  use  a  combination  approach. 

The  PBA  study  team  recommendations  were  grouped  into  five  categories:  (1)  changing 
the  risk  equation;  (2)  establishing  the  business  arrangements;  (3)  sustaining  the  PBA 
environment  after  award;  (4)  bringing  about  change  and  (5)  imique  markets.  Mitigating  risk  to 
the  point  that  fixed-price  contracts  are  acceptable  to  allocate  risk  between  the  Government  and 
industry  underpins  the  use  of  PBA.  Changing  the  risk  equation  is  paramount  in  the  DoD 
transition  to  increasing  the  use  of  PBA  (Ref  8).  As  discussed  in  the  section  on  OTA,  the  DoD 
conforms  acquisitions  to  strategy.  Commercial  firms  conform  strategies  to  acquisitions.  Broader 
use  of  PBA  by  the  DoD  will  require  a  similar  approach. 

One  of  the  recommendations  made  by  this  study  team  was  to  implement  the  use  of  multi¬ 
phase  source  selection.  The  next  section  discusses  multi-phase  source  selection  as  executed 
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today,  prior  to  the  discussion  of  the  multi-phase  source  selection  as  proposed  by  the  PBA  study 
team. 

C.  MULTI-PHASE  SOURCE  SELECTION  AS  EXECUTED  IN  TODAY’S 
ENVIRONMENT 

Source  selection  and  acquisition  strategies  as  executed  in  today’s  environment  are 
difficult  at  best  to  label.  As  discussed,  regulations  and  directives  have  been  extensively 
reworked  to  promote  flexibility,  tailoring  and  streamlining  of  acquisitions.  The  large  amount  of 
change  within  Government  acquisition  combined  with  this  promotion  of  modifying  any 
particular  strategy  to  best  fit  the  current  acquisition  has  resulted  in  a  wide  variety  of  acquisition 
strategies.  PMs  and  PCOs  administer  their  programs  from  within  this  dynamic  framework, 
drawing  from  personal  and  organizational  experiences  with  the  strategies  discussed  earlier  in  this 
Chapter. 

The  phased  down  select  is  similar  to  the  proposed  multi-phase  source  selection.  It  is  used 
here  as  a  jumping  off  point  from  which  to  look  at  an  “as  executed  today”  comparison. 

The  phased  down  select  process  has  a  number  of  similar  goals  with  the  multi-phase 
source  selection  process,  but  in  practice  has  not  been  successful.  Shared  goals  include:  (1) 
ensure  full  and  open  communications  between  the  Government  and  contractors;  (2)  maximize 
benefits  from  commimications  and  (3)  emphasize  up  front  planning.  Enhanced  communication 
is  intended  to  achieve  a  better  understanding  of  the  work  required  to  meet  Government 
requirements  and  the  risks  associated  with  contractor  accomplishment  of  this  work.  Increased 
communication  can  expand  opportunities  to  leverage  current  capabilities  or  commercial 
technology  towards  satisfying  Government  requirements.  The  end  result  is  use  of  a  fixed-price 
contract.  (Ref  8) 

The  phased  down  select  and  multi-phase  source  selection  have  two  main  differences. 
First,  in  a  phased  down  select,  refinement  of  the  requirement  and  the  entire  source  selection 
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process  is  limited  under  FAR  Part  15.  The  most  significant  FAR  Part  15  limitation  is  that 
performance,  schedule  and  cost  trade  offs  are  bounded  by  the  requirement  and  FAR  part  15. 
Requirement  changes  have  to  go  out  to  all  offerors  without  delay.  If  the  change  is  significant, 
offerors  outside  of  the  competitive  range  are  allowed  to  submit  revised  offers.  In  multi-phase 
source  selection,  changes  to  the  requirement  do  not  have  to  be  recompeted.  Changes  to  the 
requirement  may  occur  as  it  is  being  refined.  This  circumstance  could  occur  during  contract 
execution  or  in  between  contracts.  Second,  no  Justification  and  Approval  (J&A)  is  required  for 
other  than  full  and  open  competition  when  going  into  a  follow-on  phase  considering  only 
offerors  who  were  included  in  prior  phases.  This  facilitates  refinement  of  the  requirement 
without  bringing  back  offerors  excluded  in  earlier  phases. 

Multi-phase  strategies  as  executed  today  are  limited  by  these  differences.  J&As  are 
utilized  to  obtain  permission  to  limit  competition  as  the  acquisition  proceeds  jfrom  one  phase  to 
the  next.  Requirement  refinement  may  occur  due  to  maturation  or  emergence  of  technologies  or 
as  a  result  of  the  revision  of  operational  doctrine  to  accommodate  new  technology.  This 
requirement  refinement  is  oftentimes  hampered  by  inability  to  overcome  constraints  associated 
with  sharing  solutions  and  requirement  refinements  with  potential  and  current  offerors. 
Frequently  noted  in  acquisition  literature  is  that  industry  is  hesitant  to  participate  in  Government 
work  because  of  the  likelihood  of  the  source  of  their  competitive  advantage,  their  unique 
solutions,  being  disclosed  to  competitors  through  this  sequence  of  events. 

The  next  section  describes  multi-phase  source  selection  as  proposed  by  the  PBA  study 
team.  i 


D.  MULTI-PHASE  SOURCE  SELECTION  AS  PROPOSED 

One  recommendation  made  by  the  912(c)  PBA  study  group  under  the  category  of 
changing  the  risk  equation,  is  a  source  selection  strategy  labeled  multi-phase.  Per  the  PBA  study 
group,  the  multi-phase  source  selection  strategy  should  be  considered  whenever  the  requirement 
is  fluid  and  is  likely  to  change  significantly  after  information  has  been  exchanged  with  potential 
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sources  and  there  are  two  or  more  sources  expected  to  be  capable  of  meeting  Government 
requirements.  (Ref  8) 

Similar  to  several  of  the  acquisition  strategies  discussed  in  Chapter  II,  the  proposed 
multi-phase  source  selection  strategy  emphasizes  up  front  planning  and  increased 
communication  between  industry  and  the  Government.  The  proposed  multi-phase  source 
selection  strategy  is  intended  to  provide  clear  understanding  to  both  industry  and  Govenunent 
how  much  of  the  requirement  can  realistically  be  met  with  existing  technology  and  how  much 
requires  development.  Discussions  between  industry  and  Government  are  limited  to  sources 
with  high  past  performance  ratings  and  demonstrated  performance  capabilities.  There  is  no 
desire  or  intention  to  share  the  same  information  with  all  offerors.  Discussions  can  be  unique  to 
each  offeror.  Fairness  with  offerors  does  not  automatically  equate  to  sharing  the  contents  of  each 
unique  discussion  held  with  individual  offerors.  (Ref  8) 

In  multi-phase  source  selection,  Government  requirements  are  initially  stated  as 
objectives.  Offerors  provide  capability  statements  for  evaluation.  Capability  statements  may 
include  summary  information  about:  (Ref  8) 

•  Past  performance  of  the  contractor  for  similar  work 

•  Ability  to  perform  the  required  work 

•  Conceptual  approach  to  performing  the  work 

•  Rough  order  of  magnitude  price  estimate  for  the  conceptual  approach 

After  selecting  an  initial  subset  of  offerors,  the  Government  works  individually  with  each 
to  refine  requirements.  Program  risk  reduction  and  leveraging  commercial  technology  are 
inunediate  goals.  Cost,  timeline  and  capability  trade  offs  are  considered  with  each  proposed 
solution  (Ref  8).  The  first  two  phases  are  generally  proposal  development  and  requirements 
development.  A  niunber  of  phases  may  be  used.  The  desired  end-state  is  adequate 
understanding  of  requirements  by  both  the  Government  and  industry  to  propose  and  accept 
fixed-price  contracts.  Sources  may  be  eliminated  fi-om  the  competition  in  any  phase.  Criteria  to 
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compete  in  a  follow-on  phase  may  include  participation  in  the  prior  phase.  One  or  more 
contracts  may  be  awarded  for  each  phase.  Proposed  statutory  language  negates  the  need  for  a 
J&A  by  allowing  consideration  of  limited  sources  without  bringing  back  in  all  potential  offerors. 
(Ref  8) 


The  Government  may  fund  all  or  a  part  of  the  earliest  phases.  Commercial  applicability 
increases  the  likelihood  that  contractors  will  invest  in  the  development  process.  Different 
funding  levels  may  be  provided  to  each  contractor. 

Refinement  of  Government  requirements  is  expected  to  take  place  during  execution  and 
evaluation  of  the  phases.  The  Government  states  in  the  solicitation  that  it  has  the  right  to 
negotiate  the  alteration  of  requirements  or  terms  and  conditions  without  changing  the 
solicitation.  (Ref  8) 

Government  teams  can  be  formed  to  work  with  each  industry  team.  A  cost  benefit 
analysis  should  be  conducted  prior  to  forming  contractor  specific  Government  teams.  Separate 
teams  may  preclude  the  possibility  of  technical  transfusion  or  leveling.  Separate  teams  may  not 
be  achievable  due  to  lack  of  personnel  to  adequately  staff  them. 

This  source  selection  strategy  is  proposed  as  a  means  to  facilitate  using  PBA.  The  next 
section  discusses  the  relationship  between  PBA  and  multi-phase  source  selection. 

E.  RELATIONSHIP  BETWEEN  PRICE-BASED  ACQUISITION  AND  MULTI¬ 
PHASE  SOURCE  SELECTION 

Design  and  production  competition  exist  in  a  development  acquisition.  Design 
competition  occurs  in  the  early  phases  of  development.  Production  competition  occurs  after  the 
design  is  stable.  Reducing  risk  during  design  competition  has  historically  been  addressed  by 
choosing  one  design  solution  out  of  several  that  have  not  shared  data.  (Ref  9) 
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Maintaining  design  competition  carries  with  it  cost.  Choosing  one  design  and 
eliminating  this  form  of  competition  can  save  money.  (Ref  9) 

Production  competition  is  typically  longer  than  Design  competition.  Production 
competition  can  be  maintained  through  production,  fielding  and  disposal.  Production 
competition  sometimes  ends  when  one  competitor  takes  over  another.  Production  competition 
centers  on  a  stable  design  and  can  facilitate  benchmarking  between  competitors  and  thus  serve  to 
preserve  skills  in  the  industrial  base.  (Ref  9) 

Encouraging  and  sustaining  production  competition  for  complex  systems  is  not  an  easy 
undertaking.  In  some  cases  it  is  unachievable  and  not  in  the  best  interest  of  the  Government. 
Alternatives  to  sustaining  production  competition  include:  (Ref  9) 

•  Subcontract  competition 

•  Component/subsystem  breakout 

•  Use  of  the  industrial  modernization  incentive  program 

•  Aggressive  value  engineering  program 

•  Use  of  incentive  or  award  fee  contracts 

•  Should  cost  analysis  of  the  sole  source  prime 

•  Product  improvement  of  existing  item 

•  Use  of  “off  the  shelf’  commercial  or  non-developmental  items 

The  multi-phase  source  selection  strategy  is  intended  to  address  the  inherent  risks  of  both 
design  and  production  competition,  and  in  this  manner  facilitates  PBA. 

The  USD  (AT&L)  embraces  PBA,  yet  recognizes  that  it  is  a  controversial  subject. 
Therefore,  the  USD  (AT«&L)  is  looking  for  DoD  agencies  to  propose  pilot  programs  to  test  the 
recommendations  made  by  the  PBA  study  team,  including  the  multi-phase  source  selection 
strategy.  Progress  made  in  the  development  and  use  of  other  PBA  tools  such  as  waivers  of 
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certified  cost  and  pricing  data  are  highlighting  the  benefit  of  using  PBA.  Use  of  multi-phase 
source  selection  may  continue  and  accelerate  this  progress. 

PBA  is  one  approach  out  of  many  available  to  a  contracting  officer.  It  should  be 
considered  for  use  when  the  situation  warrants  as  opposed  to  using  PBA  in  all  situations. 

Properly  balancing  timelines,  cost,  requirements  and  risk  requires  flexibility.  Flexibility  is 
increased  with  inclusion  of  additional  methods  to  facilitate  PBA.  (Ref  10)  Multi-phase  source 
selection  provides  another  method  to  facilitate  PB  A  and  increase  access  to*  commercial 
technology. 

Multi-phase  source  selection  goes  beyond  what  can  be  done  xmder  existing  regulatory  and 
statutory  restraints.  Government  to  industry  and  industry  to  Government  information  flow  is 
enhanced  by  multi-phase  source  selection.  This  increased  communication  may  ensure  that  both 
Government  and  contractors  have  a  clear  and  unambiguous  imderstanding  of  the  requirement, 
industry  capability,  and  specific  solutions.  It  allows  industry  to  be  more  innovative  in  meeting 
requirements.  It  allows  industry  to  help  the  Government  tailor  requirements.  This  tailoring  can 
overcome  the  systemic  problem  of  the  Government  buying  into  optimistic  promises  from 
industry.  It  provides  a  means  to  let  competitors  be  unequal  without  requiring  equal  sharing  of 
information.  This  protects  intellectual  information.  While  this  approach  could  be  used  in  a  cost- 
based  environment,  it  is  intended  to  reduce  risk  to  the  point  where  fixed-price  contracts  are  an 
acceptable  risk  allocation  tool,  and  in  this  manner  facilitates  PBA.  (Ref  8) 

F.  CHAPTER  SUMMARY 

This  Chapter  provided  a  backdrop  of  current  and  historical  perspective  on  the 
smorgasbord  of  acquisition  and  source  selection  strategies  confronting  PMs  and  PCOs  as  they 
formulate  a  source  selection  methodology.  This  Chapter  provided  an  overview  of  the  current 
environment  in  which  system  acquisition  programs  are  immersed  prior  to  discussing  several 
programs  in  more  detail. 
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The  next  Chapter  will  summarize  four  active  acquisition  programs.  These  summaries 
will  be  used  to  draw  conclusions  about  the  use  of  multi-phase  source  selection. 
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III.  SOURCE  SELECTION  IN  SELECTED  FEDERAL  PROGRAMS 


A.  INTRODUCTION 

Four  case  studies  in  brief  are  presented  in  this  Chapter.  The  cases  were  chosen  after  a 
review  of  recent  literature  and  discussions  with  program  managers  and  support  personnel. 
Programs  utilizing  source  selection  methodology  that  included,  or  attempted  to  include  phasing 
were  sought. '  Programs  chosen  are  intended  to  highlight  innovations  achieved  in  source 
selection,  as  well  as  highlighting  barriers,  advantages,  challenges  and  disadvantages  of  each 
approach. 

The  emphasis  in  the  case  studies  is  not  on  the  history  of  each  program.  Backgroimd  is 
presented  to  give  the  reader  an  overview  only.  The  important  features  of  the  case  studies  are  the 
points  noted  by  source  selection  practitioners  during  the  execution  of  their  programs.  The  case 
studies  as  presented  in  this  Chapter  are  intended  to  provide  an  objective  overview  of  each  case. 
Further  analysis  of  points  noted  in  the  cases  will  be  presented  in  Chapter  V. 

B.  USCG  DEEPWATER  PROJECT 

The  U.S.  Coast  Guard  labels  operations  taking  place  greater  than  50  miles  from  land  the 
Deepwater  mission.  Deepwater  missions  generally  consist  of  lengthy  at  sea  operations  in  all 
weather  environments.  Missions  can  be  humanitarian,  law-enforcement,  diplomatic  or  military 
(Ref  12).  Deepwater  assets  currently  are  a  group  of  aging  legacy  systems,  at  or  past  the  end  of 
their  expected  service  life.  These  assets  were  acquired  piece-meal  over  time,  and  are  not  well 
integrated,  either  internal  to  the  Coast  Guard,  or  externally  to  other  Services  and  Government 
agencies. 

The  Deepwater  Project  was  initiated  in  1998.  Its  purpose  is  to  ensure  the  timely 
acquisition  of  the  appropriate  resources  that  utilize  available  technology  to  satisfy  Deepwater 
mission  needs.  This  purpose  is  intended  to  enable  the  Coast  Guard  to  upgrade,  modernize  and 
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replace  ships,  aircraft  and  command  and  control  infrastructure.  This  project  includes  re¬ 
capitalization  of  all  Deepwater  assets.  The  objective  is  to  develop  an  integrated  system  of 
systems.  This  system  of  systems  includes  surface,  air,  command  and  control,  communications 
and  shore-based  infrastructure  assets.  Emphasis  is  placed  on  maximizing  operational 
effectiveness  with  lowered  total  ownership  costs  over  the  life  span  of  acquired  assets.  (Ref  12) 

1.  Intended  Strategy 

The  Deepwater  acquisition  strategy  can  be  summarized  as  acquiring  an  integrated  system 
of  systems,  focused  on  capabilities  vice  assets,  with  requirements  and  solutions  defined 
collaboratively  by  both  industry  and  the  Coast  Guard  (Ref  13). 

This  acquisition  strategy  is  loosely  patterned  after  DoD  models  of  working  with 
competing  industry  teams  towards  down  selection  to  a  substantial  contract  award.  This 
acquisition  consists  of  two  phases.  Phase  I  contracts  were  for  conceptual  and  functional  design. 
Phase  II  contracts  are  to  build  and  provide  the  integrated  Deepwater  solution. 

Phase  I  award  decisions  were  based  on  an  evaluation  of  the  following  factors: 
technical/management  and  cost/price,  with  technical/management  being  significantly  more 
important  than  cost/price.  An  oral  presentation  was  used  where  offerors  addressed  project 
management  and  technical  expertise.  Written  information  evaluated  included  PPI,  software 
capability  maturity  model  assessments,  organizational  descriptions,  integrated  master  plans, 
integrated  master  schedules,  copies  of  oral  presentation  slides,  and  cost/price  information.  (Ref 
17)  The  phase  I  contracts  are  fixed-price. 

The  phase  II  contracting  strategy  is  to  award  a  single  contract  to  a  systems  integrator. 
The  systems  integrator  provides  the  system  of  systems  implementation  plan.  The  systems 
integrator  is  responsible  for  providing  a  complete  system  in  compliance  with  the  system 
performance  specification.  Phase  II  contract  type  is  intended  to  be  a  long-term  Indefinite 
Delivery  Indefinite  Quantity  (IDIQ)  contract  with  fixed-price  and  cost  type  delivery  orders. 
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Delivery  orders  may  be  structured  as  a  multi-year  contract  or  as  a  base  period  with  options. 
Performance  incentives  are  to  be  incorporated  into  the  DOs.  (Ref  16) 

Phase  II  award  decisions  will  be  based  on  an  evaluation  of  four  factors.  In  descending 
order  of  importance  they  are  operational  effectiveness,  technical,  management  and  cost/price. 
Operational  effectiveness  includes  assessments  of  effectiveness  when  implemented,  during 
implementation  and  five  and  ten  year  years  after  implementation.  Operational  effectiveness 
includes  a  measure  of  flexibility  of  the  integrated  design  system- to  changesi'in' demand:  The 
technical  factor  includes  system  performance  specification  compliance  and  functional  design 
feasibility  risk  assessments.  Management  process  includes  four  risk  assessments;  production 
plan  feasibility,  management  including  organization  and  controls,  integrated  design  systems 
integration  and  total  ownership  cost  management.  The  cost/price  factor  includes  total  operations 
and  maintenance  cost  over  the  life  cycle  of  the  solution  and  the  timeliness  of  demonstrated 
savings,  along  with  completeness  and  reasonableness  of  cost  estimates.  (Ref  17) 

This  approach  is  a  radical  departure  firom  past  acquisitions  by  the  Coast  Guard.  Industry 
is  starting  from  scratch,  working  from  broadly  stated  performance  requirements.  This  provides 
significant  leeway  in  which  to  determine  the  best  mix  of  assets  to  fulfill  the  requirement.  (Ref 

15) 


2.  Barriers  to  Use  of  Intended  Strategy 

Uncertain  funding  is  a  barrier.  This  contract  represents  the  Government  buying  a  line  of 
ships  and  aircraft  over  a  prolonged  period.  To  assist  in  development  of  proposals,  offerors  were 
provided  with  projected  budget  allocations  extending  40  years  into  the  future  (the  expected  life 
cycle  of  Deepwater).  There  is  difficulty  in  giving  assurance  to  contractors  that  future  plans  will 
be  executed.  This  inability  to  give  assurance  to  offerors  is  exacerbated  by  the  extended  length  of 
solution  implementation.  The  intended  duration  of  solution  implementation  extends  beyond  the 
authority  that  the  Coast  Guard  has  to  commit  funding. 
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Limiting  competition  when  executing  multiple  award  phases  is  another  barrier.  After  the 
first  phase,  the  Coast  Guard  deemed  it  necessary  to  limit  offerors  to  those  who  had  participated 
in  the  earlier  phase.  This  required  a  J&A  for  other  than  full  and  open  competition.  This  hairier 
was  anticipated  by  the  program  office  and  relevant  oversight  bodies.  Approval  was  given  to 
limit  competition  based  on  unique  and  proprietary  systems,  the  large  investment  by  Government 
and  industry  teams  in  the  prior  phase  and  the  fact  that  in  phase  I  the  Government  stated  that 
competition  would  be  limited  at  this  point. 

Ability  to  execute  the  strategy  within  agency  regulatory  guidelines  is  also  a  barrier.  The 
Coast  Guard  Systems  Acquisition  Manual  (SAM)  outlines  a  process  for  major  systems 
acquisitions  based  on  individual  asset  replacement  (Ref  15).  Because  of  this,  the  system  of 
systems  approach  used  in  Deepwater  does  not  mesh  with  some  of  the  document  requirements 
and  formats  prescribed.  The  Deepwater  approach  and  the  high  level  of  management  oversight 
and  attention  that  it  receives  more  than  meets  the  intent  of  SAM  procedures.  A  waiver  to  SAM 
requirements  for  phase  I  was  obtained  to  not  do  administrative  contract  management  items  like 
the  risk  management  plan  exactly  as  laid  out.  The  Deepwater  project  is  able  to  meet  SAM  intent 
without  being  restricted  by  SAM  dictated  timelines  and  formats. 

3.  Expected  Benefits  from  Use  of  Intended  Strategy 

The  main  benefit  sought  is  empowered  industry.  This  empowerment  is  intended  to  use 
field-tested  technologies  and  improved  or  new  processes  toward  the  end  of  maximizing 
operational  effectiveness  with  a  reduction  in  total  ownership  cost  (Ref  14). 

The  multiple-phase  down  select  approach  is  intended  to  motivate  cost  sharing  in  system 
development.  Contractors  were  expected  to  invest  in  the  design  phase,  above  and  beyond  the 
amount  contracted.  The  competition  during  early  design  and  development  serves  to  encourage 
innovation  and  fair  pricing.  Collaborative  teaming  between  industry  and  Government  serves  to 
reduce  overall  project  risk. 
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4. 


Execution  of  Strategy 


Phase  I,  conceptual  design,  competed  a  high-level  systems  perfonnance  requirement. 
Phase  I  source  selection  for  conceptual  design  resulted  in  the  award  of  fixed-price  contracts  to 
three  industry  teams  out  of  four  offerors. 

Phase  I  consists  of  two  parts,  conceptual  design  and  functional  design.  In  conceptual 
design  the  contractors  developed  proposed  concepts  to  about  the '50  percent  mark?'  Three  out  of  - 
four  competitors  were  awarded  phase  I  contracts  in  August  1998.  The  prime  contractors  were 
Lockheed  Martin,  Avon-Dale  Industries,  Inc.,  and  Science  Applications  International 
Corporation  (SAIC).  In  the  functional  design  phase  the  industry  teams  refined  their  concepts  to 
about  the  80  percent  mark.  Initially,  the  concept  design  phase  comprised  the  entirety  of  phase  I. 
As  concept  design  progressed.  Government  and  contractors  realized  that  an  additional  level  of 
detail  was  needed  to  proceed  further.  A  modification  to  further  develop  the  functional  design 
was  made  to  phase  I  contracts,  adding  18  months  and  $13  million  to  the  phase  I  contracts. 

The  start  of  phase  II  represents  a  competitive  decision  point.  The  Coast  Guard  intends  to 
award  one  contract  for  detailed  design  and  construction  in  January  2002.  Functional  design  ends 
in  April  2001.  The  phase  II RFP  is  on  schedule  to  be  published  following  completion  of 
functional  design.  This  provides  industry  with  proposal  preparation  time.  Proposal  evaluation  is 
scheduled  to  occur  between  July  2001  and  January  2002.  Proposals  will  be  evaluated  based  on 
how  well  they  maximize  operational  effectiveness  while  minimizing  total  ownership  cost. 

5.  Challenges  and  Advantages  of  Strategy 

The  prime  contractor  is  a  systems  integrator.  The  Government  does  not  have  privity  of 
contract  with  the  firms  actually  performing  the  manufacturing  and  design  work.  Difficulties  can 
arise  out  of  this  arrangement.  Difficulties  frequently  cited  using  this  type  of  arrangement  include 
restricted  flexibility  and  increased  time  to  incorporate  changes  and  inability  of  the  Government 
to  monitor  and  benefit  from  reduced  costs  generated  at  the  subcontractor  level.  In  the  case  of 
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Deepwater,  privity  of  contract  during  the  multiple  phases  of  system  development  is  a  challenge 
in  that  flexibility  of  design  is  desired,  as  well  as  development  of  an  optimal  solution.  Working 
with  industry  teams  through  a  systems  integrator  lengthens  communication  lines  along  with 
hindering  the  free  flow  and  active  interchange  of  ideas  and  potential  solutions. 

Addressing  configuration  management  and  logistics  support  up  front  and  early  in  system 
development  is  necessary  to  fully  evaluate  the  total  life  cycle  cost  of  the  system.  The  ability  to 
accurately  assess  this  component  of  system  cost  is  made  difficult  by  the  nature  of  the  system 
development.  Configuration  management  and  logistics  support  costs  are  difficult  to  quantify 
with  a  known  system  design.  In  the  Deepwater  acquisition  strategy,  these  costs  will  be  estimated 
and  included  in  the  evaluation  of  proposals  before  the  system  design  has  been  fully  developed. 

The  Coast  Guard  does  not  hand  off  contract  administration  to  another  agency.  In  the 
DoD,  the  Defense  Contract  Management  Agency  (DCMA)  normally  takes  over  contract 
administration  duties  of  major  systems  acquisitions  after  contract  award.  With  no  hand  off 
taking  place,  the  program  office  has  to  build  infrastructure  to  administer  the  contract.  The 
Deepwater  project  represents  a  long-term  proposition.  This  potentially  drives  the  program  office 
to  select  contract  types  that  are  less  of  a  burden  to  administer,  as  opposed  to  selecting  contract 
types  based  on  suitability  to  obtain  best  value  for  the  Government. 

The  dollar  value  of  the  conceptual  design  contract  was  such  that  the  contractor  would 
have  to  invest  in  the  development.  Both  Government  and  industry  recognized  this.  (Ref  17)  An 
outside  observer  would  tend  to  expect  the  dollar  value  of  the  contract  to  cover  the  cost  of  the 
work  specified  in  the  contract.  This  scenario  is  not  unique  to  the  Deepwater  project,  however  it 
does  represent  a  challenge  to  the  program. 

An  advantage  achieved  is  that  industry  has  been  empowered  by  this  source  selection 
strategy.  Solutions  proposed  have  been  outside  the  expected  norms  of  systems  developed 
utilizing  traditional  acquisition  strategies  (Ref  18).  An  increasingly  common  way  to  refer  to  the 
goal  of  industry  combining  new  commercial  and  military  technologies  in  a  unique  solution  is  to 
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label  it  as  leveraging  commercial  practices  and  technologies  (Ref  6, 25).  In  the  case  of 
Deepwater,  this  advantage  appears  to  have  been  achieved. 

C.  ARSENAL  SHIP 

In  the  early  1990s,  the  Navy  experienced  a  growing  concern  for  integrating  support  of 
littoral  warfare  in  its  weapon  systems.  Conceptually,  a  missile  barge  was  envisioned  to  support 
theater  airtiefense  and  land  battles  in  the  littoral  region.  The  requirement  was  loosely  articulated 
as  a  need  to  launch  a  significant  amount  of  precision-guided  munitions  ashore.  (Ref  6) 

The  Arsenal  Ship  was  a  different  operational  concept.  The  organizational  structure  and 
business  processes  were  a  departure  from  the  norm  in  military  shipbuilding.  This  program  was 
run  by  DARPA.  After  the  High  Altitude  Unmanned  Aerial  Vehicle  Program,  Arsenal  Ship  was 
the  second  time  DARPA  utilized  10  U.S.C.2371,  as  supplemented  by  Section  845  of  Public  Law 
103-160,  OTA  in  an  acquisition  (Ref  2).  OTA  was  used  to  gain  flexibility  in  negotiating  terms 
and  conditions  appropriate  for  the  acquisition. 

At  the  start  of  the  program  three  goals  were  stated;  first,  to  affordably  demonstrate 
capability;  second,  to  leverage  commercial  practices  and  technologies;  and  third,  to  demonstrate 
the  reformed  acquisition  process.  (Ref  6) 

1.  Intended  Strategy 

The  Arsenal  Ship  program  had  six  phases.  Phase  I  was  six  months  long.  During  phase  I, 
cost-performance  studies  to  support  a  preliminary  concept  for  design  were  performed.  Phase  II 
was  12  months  long.  During  phase  II,  proposed  concepts  would  be  developed  into  a  functional 
design.  Phase  III  was  33  months  long.  During  phase  III,  detailed  design  and  construction  of  a 
demonstrator  was  to  be  accomplished.  Phase  IV  was  a  one  year  long.  In  Phase  IV,  test  and 
evaluation  would  be  performed  to  demonstrate  military  utility,  ship  capabilities  and  concept  of 
operations.  Phase  V  would  be  options  for  production  of  five  ships  and  conversion  of  the 
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demonstrator  to  the  production  configuration.  Phase  VI  would  be  a  flexible  option  for  service 
life-cycle  support  tasks  for  the  proposed  fleet  of  six  Arsenal  Ships.  (Ref  6) 

Flexibility  gained  through  the  use  of  OTA  was  to  allow  the  program  to  encompass  the 
approach  of  having  the  contractors  demonstrate  operational  vice  engineering  performance.  The 
contractors  would  be  responsible  for  design,  as  opposed  to  the  Government  shouldering  this 
responsibility.  The  Government  did  not  specify  detailed  requirements  or  specifications.  The 
goal  of  this  methodology  was  to  increase  solution  space  and  accommodate  new  or  unique 
conceptual  designs.  (Ref  6) 

The  strategy  employed  used  these  principles:  (Ref  6) 

•  Use  of  relatively  few  broad  performance  goals  to  describe  desired  system  capabilities 

•  Giving  full  design  responsibility  to  the  competing  contractor  teams,  and  facilitating 
this  via  excluding  Govemment-Fumished-Equipment  (GFE)  from  the  program 

•  Use  of  a  small  joint  program  office 

•  Designating  affordability  as  the  only  requirement,  and  putting  an  emphasis  on  a  small 
crew  size 

•  Structuring  a  streamlined  program  and  processes  using  OTA 

Source  selection  was  different  in  each  phase.  The  phase  I  solicitation  outlined  the 
Government’s  vision  of  the  entire  program.  Schedule  and  funding  profiles  were  provided  for 
phases  I  through  IV  in  the  solicitation.  Offerors  were  asked  to  propose  unique  approaches 
intended  to  best  meet  Government  desired  capability.  In  their  proposal,  offerors  had  to  include 
an  agreement  that  would  serve  as  the  contractual  vehicle  for  developing  their  approach.  This 
element  of  the  acquisition  strategy  differed  sharply  from  the  traditional  shipbuilding  method  of 
the  Government  specifying  a  contract  to  be  negotiated.  (Ref  6) 
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Phase  I  awards  were  based  on  five  elements:  (Ref  6) 


•  Demonstrated  understanding  of  capabilities  document,  concept  of  operations  and 
program  description  including  processes  required  to  execute  the  program 

•  Demonstrated  capability,  experience  and  resources 

•  Integrated  master  schedule  depiction  of  a  realistic,  time  phased  plan  to  achieve  phase 
I  efforts 

•  Addressed  or  demonstrated  intent  to  use  innovativd  busirie'ss  and  technical  concepts 
leading  to  reductions  in  cost  and  schedule  throughout  the  program 

•  Fit  between  amount  of  effort  proposed  in  the  task  description  for  phase  I  and 
proposed  costs  for  phase  I 

Offerors  had  to  submit  data  for  phase  II  four  months  after  phase  I  began.  This  data 
submission  consisted  of  formal  design  concepts  and  oral  presentations.  Since  phase  I  was  only 
six  months  long,  there  was  no  opportunity  for  contractors  to  use  the  full  output  of  phase  I  in 
developing  phase  II  proposals.  A  formal  question  and  answer  period  was  not  provided  for  and 
Best  and  Final  Offers  (BAFO)  were  not  taken.  (Ref  6) 

Multidisciplinary  evaluation  teams  augmented  with  Government  experts  from  the 
acquisition  community  were  used  to  evaluate  offers.  Cost  experts  were  on  the  technical 
evaluation  teams  and  technical  experts  were  on  the  cost  evaluation  teams. 

The  phase  II  down  select  was  predicated  on  choosing  the  best  value  contractor.  Down 
select  criteria  included  how  well  the  team  described  a  credible  development  program,  how  well 
the  concept  demonstrated  mission  capability  while  minimizing  life  cycle  costs  and  demonstration 
of  the  ability  to  execute  phases  II-IV  within  the  amounts  specified  in  the  funding  profile. 
Proposals  were  judged  in  whole,  with  evaluation  criteria  that  was  not  ranked  or  weighted.  (Ref  6) 

Contractors  developed  functional  designs  for  the  majority  of  phase  II.  The  Arsenal  Ship 
program  was  cancelled  two  weeks  before  phase  III  proposals  were  due.  This  cancellation 
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occurred  as  a  result  of  a  number  of  converging  circumstances.  These  circumstances  included 
overlapping  development  of  future  surface  combatants  in  other  programs,  higher-level  concerns 
about  funding,  and  shifting  priorities  bounded  by  funding  constraints  within  the  Department  of 
the  Navy. 

The  phase  III  down  selection  was  initiated  with  two  draft  solicitations.  The  program 
office  met  individually  with  each  contractor  team  as  part  of  refining  the  draft  solicitation. 
Normally  not  allowed  under  the  FAR,  the  individual  meetings  and  discussions  with  contractor  ■  - 
teams  addressed  proprietary  design  issues.  The  lack  of  an  appropriate  forum  in  which  to  address 
proprietary  design  issues  under  traditional  acquisition  processes  generally  results  in  this  type  of 
issue  being  left  off  the  table.  (Ref  6) 

Phase  III  would  use  a  cost-reimbursable  agreement  and  incentive  fee  structure.  Cost 
control  and  milestones  would  be  negotiated.  The  selected  contractor’s  agreement  would  be 
amended  to  include  phase  III  requirements.  Offerors  were  responsible  for  setting  up  milestones 
and  payments  in  their  proposed  agreement  amendments.  Negotiating  incentive  structure  details 
before  the  down  select  maintained  competition  between  offerors. 

2.  Barriers  to  Use  of  Intended  Strategy 

Phase  III  was  underfunded  (Ref  6).  Prior  to  the  start  of  the  program,  analysis  was 
conducted  to  estimate  costs  and  other  resources  needed  for  the  program.  This  analysis  did  not 
provide  a  complete  enough  understanding  on  which  to  base  sound  estimates.  The  magnitude  of 
the  development  work  was  not  adequately  understood  at  the  start  of  the  program.  This  resulted 
in  underestimating  the  amount  of  work  actually  required  to  develop  the  system.  Contractors 
were  willing  to  invest  in  the  first  two  phases  only.  During  phase  II,  further  understanding  of 
tasks  required  to  develop  the  system  resulted  in  an  expansion  of  development  tasks  required. 
The  contractors  concluded  that  Congress  would  not  provide  additional  funding  for  the  program 
beyond  that  which  was  initially  provided  for  in  phase  III.  Since  the  estimates  for  phase  III  were 
now  understood  to  be  low,  trade  space  between  cost  and  performance  shrank.  Capabilities  and 
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performance  enhancing  components  were  eliminated  from  the  demonstrators  to  cut  costs.  The 
end  result  of  this  sequence  of  events  would  be  demonstrators  that  did  not  fully  portray  the 
potential  of  the  weapon  system  and  the  possibility  of  losing  external  support  for  the  program. 

The  Arsenal  Ship  program  method  of  doing  business  outside  of  the  traditional  acquisition 
process  created  difficulties  when  interfacing  with  other  Navy  activities.  DARPA  managed  the 
first  two  program  phases,  as  opposed  to  the  Navy  managing  them.  Naval  Surface  Warfare 
Center  (NSWC)  labs  and  Navy  Participating  Managers  (PARMs)  were  reluctant  to  provide 
information  and  equipment  access  to  contractor  teams,  a  practice  they  were  not  accustomed  to. 
Lack  of  command  authority  in  DARPA  over  these  activities  exacerbated  this  difficulty.  (Ref  6) 

Steadfast  adherence  to  the  plan  as  envisioned  at  its  inception  limited  flexibility  during 
execution  (Ref  6).  During  phase  I,  the  possibility  of  moving  the  program  from  DARPA  to  Navy 
management  arose.  Contractors  pictured  a  shift  to  more  traditional  acquisition  methods  and 
stated  that  they  would  not  be  able  to  provide  the  demonstrators  under  this  approach.  The 
program  office  maintained  the  original  plan  and  funding  profile.  The  program  was  merged  with 
a  more  defined  class  of  ships,  the  SC-21,  which  was  being  developed.  Now  the  Arsenal  Ship 
would  be  a  demonstrator  only,  with  no  production.  Delivery  dates  were  extended  beyond  the 
point  of  acceptability  to  the  contractors.  At  the  close  of  phase  II,  the  contractors  were  required  to 
submit  an  irrevocable  Unit  Seiil-away  Price  (USP).  As  the  development  work  was  expanded,  the 
contractors  saw  the  establishment  of  an  irrevocable  USP  as  limiting.  In  phase  II,  the  contractors 
allocated  costs  to  the  USP  based  on  how  well  each  functional  area  could  justify  its  estimate.  The 
best  understood  (and  therefore  the  lowest  risk)  areas  received  the  funds  they  requested.  The  least 
understood  (and  therefore  the  highest  risk)  areas  received  the  funds  left  over  (Ref  6).  This 
method,  intended  to  address  CAIV,  improperly  allocated  the  cost  goal  and  diminished  system 
performance  after  tradeoffs  were  made. 

All  contractors  did  submit  an  irrevocable  USP.  Some  of  the  contractors  speculated  that 
this  USP  would  not  be  irrevocable  per  se.  These  contractors  felt  that  submitting  an  irrevocable 
USP  without  completing  detailed  design  was  impractical.  As  the  development  work  had 
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changed  or  expanded  in  the  first  two  phases,  these  contractors  foresaw  further  design  changes 
occurring  in  the  next  two  phases.  These  design  changes  had  the  potential  to  undennine  the 
applicability  of  the  irrevocable  USP.  By  submitting  an  irrevocable  USP  before  completing 
detailed  design,  production  price  would  be  set  without  a  detailed  design.  Investment  in  the 
project  made  by  contractors  in  phase  III  could  not  be  earned  back  in  production. 

During  phase  II,  one  contractor  noted  that  certain  requirements  resulted  in  fixed-price 
development^'  an  undesirable  outcome.  Fixed  phase  III  fimding  combined  with  an  irrevocable  ■  ' 
offer  for  phase  V  and  a  technical  matrix  attached  to  the  phase  III  solicitation  led  to  this  claim. 
Technical  information  for  all  design  elements  had  to  be  indexed  per  the  matrix.  Detailed 
accounting  was  required  for  the  complete  system.  This  linkage  between  the  solicitation, 
technical  information  and  detailed  accounting  was  in  essence  a  design  specification.  All 
contractor  teams  added  change  clauses  to  their  proposed  phase  III  agreement  modifications  to 
make  the  irrevocable  offer  non-binding.  The  program  office  maintained  that  this  combination  of 
program  requirements  was  not  fixed-price  R&D  as  long  as  contractors  could  make  trades  in 
performance  within  the  cost  constraints.  Not  holding  the  contractor  selected  to  the  system 
specified  in  the  technical  matrix  would  preclude  fixed-price  development. 

Prior  to  the  start  of  phase  II,  phases  I  and  II  were  noted  as  being  of  insufficient  length  by 
the  Government.  Phase  I  was  originally  scheduled  for  six  months  and  phase  II  was  originally 
scheduled  for  12  months.  At  this  juncture,  the  program  office  would  have  preferred  nine  and  18 
months  for  phases  I  and  II  respectively.  Contractors  were  required  to  submit  written  phase  II 
proposals  four  months  after  phase  I  contracts  were  awarded.  In  the  space  of  four  months  the 
contractors  had  to  do  mission  analysis,  derive  performance  requirements,  develop  major  design 
options  and  reasonably  understand  USP.  Because  of  the  time  limitation,  these  processes  were 
done  in  parallel.  The  output  was  not  completed  with  sufficient  detail  and  consistency.  This 
stage  of  a  development  program  drives  the  majority  of  cost  over  the  life  of  the  program.  More 
effort  devoted  at  this  juncture  would  get  the  most  return  on  dollars  spent  overall.  Contractors  in 
this  phase  were  comfortable  with  the  schedule,  even  though  parallel  completion  of  tasks  was 
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required.  An  example  of  parallel  tasking  was  the  advance  fimding  of  subsequent  phases  to 
obtain  supplier  bids.  (Ref  6) 

The  procurement  regulations  of  a  traditional  acquisition  program  were  seen  as  a 
significant  barrier  to  streamlining  the  Arsenal  Ship  acquisition.  Hand-in-hand  with  procurement 
regulations  is  the  barrier  of  the  mindset  of  the  participants  in  the  process  (Ref  21).  As  an 
example,  in  a  traditional  military  shipbuilding  acquisition,  the  Navy  takes  its  contract  design  and 
develops  an  RFP.  The  RFP  includes  detailed  design  and  construction  of -the- lead  ship,  and 
possibly  options  for  follow-on  ships.  The  RFP  is  usually  comprised  of  a  group  of  written 
volumes  hundreds  or  thousands  of  pages  long.  Contractors  invest  millions  of  dollars  and  several 
months  preparing  proposals.  A  question  and  answer  period  follows  the  receipt  of  proposals  by 
the  Government.  A  revised  RFP  may  be  issued,  after  which  the  cycle  is  repeated.  A  final 
proposal  revision  is  eventually  requested  from  the  offerors,  and  the  Government  awards  a 
contract.  This  process  has  taken  anywhere  from  six  months  to  two  years  to  complete.  (Ref  21) 

3.  Expected  Benefits  from  Use  of  Intended  Strategy 

Additional  insight  provided  to  contractors  was  an  expected  benefit.  Detailed  debriefs  to 
successful  offerors  conducted  over  multiple  phases  would  provide  industry  teams  full  insight  into 
what  the  Government  saw  as  their  strengths  and  weaknesses.  This  feedback  would  reduce 
program  risk  by  improving  contractor  effort  in  subsequent  phases.  (Ref  21) 

Shortened  somce  selection  schedule  was  an  expected  benefit.  Continuous  interaction 
with  industry  teams  during  each  phase  would  shorten  the  source  selection  schedule.  The  tight 
schedule  was  a  significant  challenge.  A  tremendous  amount  of  information  had  to  be  processed 
with  detailed  analysis  required  in  short  order.  Ongoing  discussions  with  contractors  about 
technical,  cost  and  management  approaches  in  the  early  phases  would  result  in  frequent 
reassessment  of  risk. 
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Streamlined  process  was  an  expected  benefit.  The  use  of  OTA  would  eliminate  most  of 
the  procurement  regulations.  This  allowed  streamlining  of  the  process  and  shortening  of  the 
schedule,  especially  the  source  selection  process. 

Protest  avoidance  was  an  expected  benefit.  OTA  was  utilized  as  a  means  to  remove 
“protest  avoidance”  from  the  process  (Ref  21).  The  program  office  diligently  pursued 
maintenance  of  the  integrity  of  offeror  designs  and  business  data.  With  OTA,  the  requirement  to 
provide  answers  to  one  offeror’s  questions  to  all  offerors  would  not  exist.  This  improved  the 
ability  of  the  Government  to  support  and  communicate  with  individual  industry  teams. 

Using  OTA  to  change  the  mindset  of  the  participants  was  an  expected  benefit.  The 
traditional  acquisition  process  reinforces  risk-averse  business  practices,  and  can  inhibit 
innovation.  Using  OTA  allows  the  removal  of  the  traditional  acquisition  process  in  its  entirety. 
Regulations  and  business  practices  can  then  be  examined  individually  and  reinserted  on  a  case- 
by-case  basis,  adjusted  to  the  acquisition.  Examples  applicable  to  the  Arsenal  Ship  program  are 
regulations  governing  security  and  explosive  safety.  (Ref  21) 

Increased  design  innovation  was  an  expected  benefit.  Allowing  industry  to  set 
requirements  would  increase  design  innovation.  Industry  had  free  use  of  the  available  trade 
space  for  design  and  subsystem  selection.  This  facilitated  the  use  of  new  technologies  and 
innovative  integration  of  technologies. 

4.  Execution  of  Strategy 

The  Arsenal  Ship  program  started  18  March  1996.  Six  industry  teams  competed  for 
phase  I  work.  In  July  1996,  five  of  the  six  industry  teams  were  awarded  $1  million  agreements 
for  phase  I. 

In  January  1997,  three  of  the  five  industry  teams  were  awarded  $15  million  agreements 
for  phase  II.  The  plan  had  been  to  down  select  to  two  industry  teams  for  phase  II.  The  third 
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industry  team  was  retained  in  the  competition  as  a  means  to  include  unique  expertise  and 
technologies  they  brought  to  the  table.  The  additional  $15  million  R&D  investment  to  maintain 
the  third  competitor  in  phase  II  was  not  insignificant  to  the  program  office,  but  was  not  seen  as  a 
significant  issue  when  contrasted  with  the  planned  production  schedule  or  the  DoD  R&D  budget. 

Phases  I  and  II  were  completed  as  originally  scheduled.  The  program  was  cancelled  20 
months  after  it  started. 

5.  Challenges  and  Advantages  of  Strategy 

The  contractors  were  expected  to  (and  did)  buy  in  to  the  first  two  phases.  Each  of  the 
five  phase  I  contractors  invested  about  $5  million  for  a  total  of  $25  million.  Each  of  the  three 
phase  II  contractors  invested  approximately  the  same  amount  the  Government  did  ($15  million) 
for  a  total  of  $45  million  (Ref  6). 

Stated  earlier  as  a  barrier  to  use  was  the  issue  of  insufficient  funding.  The  factors 
underlying  the  insufficient  funding  caused  a  constraint  to  trade  space.  This  constraint  of  trade 
space  represented  a  significant  challenge  in  this  program. 

Interaction  between  the  elements  of  a  strategy  can  be  unpredictable  and  complicated. 

This  interaction  can  be  supportive  or  a  source  of  unhealthy  conflict.  In  the  case  of  the  Arsenal 
Ship  program,  the  small  program  office,  OTA  and  minimal  system  specification  worked  together 
to  add  flexibility  to  the  program.  The  notion  that  an  iimovative  strategy  should  be  implemented 
as  a  package  (Ref  6)  represents  both  a  challenge  and  an  advantage  in  this  case. 

As  competition  is  eliminated  throu^  down  selects,  the  relationship  between  the 
contractor  and  the  Government  changes.  With  no  competitors  immediately  present,  a  sole 
contractor  is  not  directly  compelled  to  put  forth  effort  to  remain  amenable  with  the  Government 
(Ref  6).  Barriers  to  entry  of  competitors  is  significantly  raised  at  this  juncture  and  the  remaining 
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contractor  has  the  potential  to  “tell  it  like  it  is”  as  opposed  to  worrying  more  about  painting  a 
more  optimistic  picture  than  competitors. 

Four  advantages  were  achieved  with  the  strategies  utilized  in  the  Arsenal  Ship  program: 

•  Leveraging  of  commercial  practices  and  technologies 

•  Minimized  size  of  program  office 

•  Improved  contractor  efforts  as  a  result  of  continuous  interaction  and  debriefs  from  the 
program  office 

•  Shortened  source  selection  process 

D.  MARINE  CORPS  AVIATION  SYSTEM  MASTER  PLAN  (MCASMP) 

The  MCASMP  program  was  established  to  procure  aviation  simulators.  These  simulators 
are  for  Marine  Corps  Aviation  Assets  such  as  the  KC-130,  AV-8B,  EA-6B,  AH-1 W,  ATC,  UH- 
IN,  CH-46,  CH-53,  and  F/A-18.  The  end  user  is  the  Naval  Air  Warfare  Center  Training  Systems 
Division  (NAWCTSD).  Approximately  ten  of  the  simulators  are  considered  to  be  original 
pattern  devices,  or  the  first  unit  for  that  device  type.  The  simulators  are  to  be  self  contained  and 
readily  deployable  in  standard  containers. 

The  objective  of  the  MCASMP  is  to  develop  capability  in  simulation,  for  post  Fleet 
Readiness  Squadron  (FRS)  training.  The  simulators  will  be  used  to  immerse  experienced  pilots 
in  a  realistic  flight  environment  with  all  its  visual  and  mental  stressors.  The  simulators  are  to 
provide  realistic  simulation  of  actual  conditions  with  the  exception  of  motion  and  vibration,  and 
will  link  with  training  devices  for  other  platforms  during  coordinated  exercises.  All  simulators 
will  be  interoperable  within  their  respective  networked  environments  so  missions  can  be 
combined  as  they  would  be  in  a  combat  operation. 

Integrated  Logistics  Support  (ILS)  to  operate  and  maintain  life  cycle  support  for  the 
simulators  is  included  in  the  procurement.  ILS  includes  a  number  of  items.  Included  are  spare 
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parts,  tools  and  test  equipment,  technical  documentation,  configuration  and  inventory 
management,  an  embedded  self-paced  instructor-training  program,  and  one  year  of  Contractor 
Field  Services  for  each  type  device.  The  Contractor  Field  Services  are  to  solve  software 
problems,  assist  in  making  difficult  repairs,  and  provide  operation,  maintenance  and  support 
training.  Contractors  are  encouraged  to  pursue  life  cycle  support  initiatives  in  all  areas  that  will 
result  in  greater  operational  efficiencies  and  reduce  life  cycle  cost.  The  life  cycle  for  devices  is 
estimated  to  be  fifteen  years. 

■ ..  i 

Minimization  of  Total  Ownership  Cost  is  also  stated  as  a  primary  goal.  Contract  award 
was  originally  scheduled  for  February  2000 

1.  Intended  Strategy 

The  program  office  was  initially  faced  with  developing  a  system  with  funds  committed 
and  no  well-defined  user  requirements.  Headquarters  Marine  Corps  was  content  to  turn  the 
development  effort  over  to  Government  laboratories  (Ref  20).  This  was  not  considered  an  option 
by  the  program  office  due  to  0MB  Circular  A-76  outsourcing  requirements.  The  program  office 
saw  a  conflict  between  the  three  potential  stakeholders  in  systems  development.  The 
Government  labs  were  accustomed  to  doing  spiral  development  in  a  time  and  materials  type  of 
environment.  Government  procurement  agencies  were  accustomed  to  level  of  effort 
developmental  efforts  with  their  role  being  one  of  an  inspector  of  completed  products.  Industry 
was  poised  to  execute  an  exact  solution  as  proposed  by  the  Government. 

The  program  office  desired  to  mutually  evolve  the  user’s  requirements  in  a  dedicated 
analysis  and  definition  phase.  Participants  included  the  users,  industry  and  the  program  office. 
Their  desire  was  to  solve  the  problem  of  having  the  winning  industry  offeror’s  proposal  “become 
gospel”  before  user  requirements  were  fully  imderstood  (Ref  19).  The  program  office  saw  this  as 
requiring  several  phases,  the  first  of  which  was  a  training  systems  analysis  and  definition  phase. 
Exit  criteria  for  this  phase  would  be  development  of  three  items:  a  sponsor  user  agreement;  a  cost 
and  schedule  baseline  and  a  system  performance  specification. 
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The  sponsor  user  agreement  was  to  fully  define  and  understand  the  requirement.  The 
cost  and  schedule  baselines  were  intended  to  baseline  existing  processes  and  perform  a  gap 
analysis  to  define  the  differences  between  existing  processes  and  those  needed  to  successfully 
execute  system  development  and  implementation.  The  optimal  outcome  of  this  gap  analysis 
would  be  a  melding  of  the  best  of  Government  and  industry  practices  into  a  set  of  common 
processes.  The  system  performance  specification  was  a  statement  of  work  developed  from  the 
initial  statement  of  objectives  provided  by  the  Government. 

The  program  office  saw  the  training  system  analysis  and  design  phase  as  a  means  to 
expand  the  design  space  in  their  program.  System  cost  and  performance  tradeoffs  needed  to  be 
accommodated  early  on,  but  downstream  knowledge  had  to  be  brought  back  to  earlier  phases  of 
design  to  make  knowledgeable  system  level  tradeoffs. 

The  methodology  chosen  was  to  use  an  IDIQ  contract  with  award  to  a  single  contractor 
for  an  initial  delivery  order  encompassing  the  training  systems  analysis  and  definition  phase. 

This  delivery  order  is  to  be  followed  by  delivery  orders  to  build,  test  and  field  initial  pattern  units 
and  eventually  production  units. 

2.  Barriers  to  Use  of  Intended  Strategy 

Use  of  PPI  as  a  true  discriminator  in  source  selection  was  a  barrier  to  use.  PPI  was 
provided  by  each  offeror,  as  well  as  by  Government  sources.  The  PPI  could  be  interpreted  as 
positive  or  equally  negative  for  all  contractors.  This  equality  or  ambiguity  in  interpretation 
rendered  the  use  of  PPI  as  a  decision  tool  nearly  useless.  (Ref  19, 20) 

Money  at  risk  limited  the  ability  to  re-compete  contracts  after  delivery  of  the  sponsor  user 
agreement.  The  MCASMP  program  was  initiated  by  funding  being  committed  to  the  program 
(Ref  20).  This  placed  a  severe  constraint  on  the  program  manager  to  commit  funds  before  they 
expired.  Development  of  the  requirement,  referred  to  here  as  the  sponsor  user  agreement,  took  a 
finite  period  of  time  to  develop  to  the  point  of  being  able  to  progress  forward  with  systems 
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development.  The  nature  of  the  requirement  required  evolution  of  the  requirement  during  its 
definition. 

Use  of  open  systems  architecture  in  a  software  intensive  program  was  a  barrier  to  use. 
Particularly  in  image  databases,  achieving  a  non-proprietary  solution  is  unachievable  (Ref  20). 
This  limitation  caused  considerable  conflict  between  achieving  program  goals  of  open  systems 
throughout  with  contractor  desires  to  maintain  control  over  licensing,  rights  and  other  firms 
access  to  their  primary  means  of  maintaining  competitive  advantage  in  the-  marketplace. 

Resolving  source  of  labor  funding  in  a  joint  contractor/Govemment  DPT  environment  was 
a  barrier  to  use.  Two  budgets  existed  in  the  IPT,  Government  labor  and  contractor  labor.  This  is 
a  Government  institutionalized  way  of  doing  business  (Ref  20).  The  program  office  considered 
turning  Government  funding  sources  over  to  the  contractor,  and  treating  Government  employees 
like  another  subcontractor.  This  didn’t  work  out  due  to  an  inherent  conflict  of  interest.  The 
conflict  of  interest  resulted  from  a  CPDF  contract  where  the  person  evaluating  the  incentive  (the 
Government  employee)  also  was  getting  paid  on  it.  This  is  the  same  as  saying  to  industry,  “we 
are  paying  for  this  Government  labor  no  matter  how  much  you  use.  You  are  in  a  CPIF  contract 
incentivized  to  reduce  cost.  How  much  Government  labor  do  you  need?”  The  rational 
contractor  answer  is  “as  much  as  you  can  provide  (Ref  20).”  The  method  used  to  overcome  this 
difficulty  is  to  use  an  Earned  Value  Management  System  (EVMS)  to  map  Government  monies  to 
the  Statement  of  Work  (SOW)  and  Work  Breakdown  Structure  (WBS). 

Ongoing  requirement  and  solution  refinement  during  execution  of  contracts  hampered 
Government  ability  to  compete  contracts.  Using  an  IDIQ  contract  where  the  contractor  develops 
the  SOW  does  not  tie  the  Government  to  the  SOW.  With  the  SOW  generated  by  the  contractor 
the  contention  is  that  the  requirement  does  not  change  as  long  as  the  initial  SOO  is  sufficiently 
broad-based.  (Ref  19, 20) 
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3. 


Expected  Benefits  from  Use  of  Intended  Strategy 


The  IDIQ  will  allow  the  Government  the  flexibility  of  awarding  follow-on  production 
units,  via  issuance  of  delivery  orders,  on  a  Cost  Plus  Incentive  Fee  (CPIF),  FPI  or  FFP  basis. 

The  pricing  basis  for  each  simulator  will  be  determined  based  upon  the  availability, 
completeness,  and  applicability  of  technical  and  cost  data  for  these  or  similar  units.  The  goal  is 
to  move  to  a  FFP  arrangement  for  production  simulators  as  soon  as  adequate  configuration 
baseline  stabilization  is  achieved.  (Ref  19)  •  •  . 

The  program  office  envisioned  an  Indefinite  Delivery/Indefinite  Quantity  (IDIQ)  type  of 
contract.  This  type  of  contract  is  intended  to  allow  flexibility  in  defining  the  scope  of  work 
required  for  follow-on  pattern  and  production  units  and  to  assure  technical  compliance  and 
affordability.  This  also  facilitates  appropriate  contract  type  selection  for  each  simulator 
acquisition,  depending  on  the  stage  of  development.  It  also  allows  for  the  adjustment  of  types 
and  quantities  of  simulators  acquired  during  the  eight-year  period  of  performance. 

The  incremental  commitment  of  funds  would  serve  to  mitigate  risk.  Evolution  of 
requirement  in  this  software  intensive  business  frequently  leads  the  program  office  into  the 
position  of  having  to  determine  if  the  “contractor  can  do  it.”  Mitigation  is  accomplished  through 
commitment  of  funds  in  increments  via  delivery  orders  tied  to  discrete  elements  of  the  overall 
acquisition. 

4.  Execution  of  Strategy 

A  SOO  outlined  the  Government's  requirements.  The  contractor  generated  the  Statement 
of  Work,  specification  and  the  necessary  Contract  Data  Requirements  List.  The  original  pattern 
unit  for  each  device  is  to  be  designed  and  constructed  by  a  fully  collocated  Integrated  Product 
Team  (IPT),  consisting  of  the  successful  offeror  and  various  Government  agencies  and  their 
representatives.  The  successful  contractor  then  becomes  responsible  for  the  construction,  testing, 
shipping,  and  installation  of  any  follow-on  devices. 
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The  pattern  unit  simulators  will  be  designed  utilizing  spiral  development.  The  devices 
are  to  simulate  the  design  basis  aircraft  per  the  priorities  laid  out  in  the  SOO.  The  contractor  is 
expected  to  maximize  commonality  across  all  trainer  platforms.  The  training  devices  are 
required  to  be  open  and  non-proprietary.  Maximizing  use  of  commercial  and  Non- 
Developmental  Items  is  emphasized. 

The  program  office  seeks  an  innovative  solution  to  meet  the  requirements  of  the 
MCASMP  acquisition.  A  competitive  contract  award  will  be  made  based  on>  a  trade-off 
evaluation.  An  IDIQ  contract  will  be  awarded  to  the  responsible  offeror  whose  offer  is 
determined  to  provide  the  best  combination  of  technical  approach,  past  performance,  cost  and 
subcontracting  plan.  The  trade-off  evaluation  approach  is  intended  to  allow  the  SSA  flexibility 
in  selecting  the  proposal  that  best  meets  Government  requirements. 

A  SOO  was  provided  to  the  offerors  as  part  of  the  RFP.  The  Government  requested 
technical  (including  ILS  and  program  management),  past  performance,  and  cost  proposals  from 
interested  offerors.  The  subcontracting  plan,  required  from  competing  large  businesses,  is  also  a 
trade-off  factor.  Each  offeror's  technical  proposal  consisted  of  a  SOW  and  recommended 
contract  data.  The  technical  and  past  performance  proposal  areas  are  equally  important, 
followed  by  cost  and  the  subcontracting  plan  in  descending  order  of  priority. 

Oral  presentations  were  used  for  the  presentation  of  the  program  management  portion  of 
the  technical  proposal.  Oral  presentations  were  to  demonstrate  the  offerors’  proposed  IPT 
structure,  including  but  not  limited  to  specific  utilization  of  Government  resources  and  a 
transition  plan  addressing  how  the  IPT  structure  would  accommodate  concurrent  initial  pattern 
unit  development  at  Government  and  offeror  facilities. 

For  the  technical  factor  a  proposal  rating  and  proposal  risk  were  assessed.  The  proposal 
rating  depicted  how  well  the  proposed  approach  met  solicitation  requirements.  The  technical 
evaluation  for  the  second  delivery  order  was  an  evaluation  of  the  system  design  proposed  for  the 
second  delivery  order.  The  proposal  risk  addressed  potential  impacts  of  the  proposed  approaches 
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on  performance,  schedule,  and  cost  in  achieving  solicitation  requirements  and  program 
objectives.  For  past  performance,  a  past  performance  risk  assessment  was  made.  PPI  included 
subcontractor  and  team  member  relevant  demonstrated  past  performance  and  systemic 
improvement.  Offerors  not  required  to  submit  a  subcontracting  plan  were  given  a  “satisfactory” 
rating  in  the  subcontracting  plan  evaluation  area. 

If  an  offeror  had  high  past  performance  risk,  past  performance  became  a  key  selection 
criterion.  To  encourage  large  businesses  to  expand  subcontracting  opportunities  for  small  and 
disadvantaged  businesses,  a  small  business  plan  evaluation  factor  was  included  to  increase  the 
trade-off  rating  for  subcontracting  goals  to  small  and  disadvantaged  firms. 

A  formal  source  selection  process  was  used  for  this  program.  A  Source  Selection 
Evaluation  Board  (SSEB)  evaluated  the  technical  proposals,  cost/price,  past  performance,  and 
subcontracting  plan  and  provided  its  finding  to  a  Source  Selection  Advisory  Council  (SSAC). 
The  SSAC,  consisting  of  senior  members  of  each  competency,  reviewed  the  SSEB  input  and 
performed  a  comparative  analysis  among  the  proposals  received,  providing  a  proposal  analysis 
report  to  the  SSA.  The  SSA  determined  the  offeror  to  award. 

Delivery  Orders  (DO)  issued  under  the  IDIQ  contract  will  be  priced  on  a  CPIF  basis  with 
incentive  being  placed  on  cost  only.  These  DOs  will  encompass  the  training  systems  analysis 
and  definition  effort  and  the  production  of  the  first  four  initial  pattern  units.  A  CPIF  pricing 
arrangement  is  considered  necessary  due  to  technical,  schedule,  and  cost  risks.  These  technical, 
cost,  and  schedule  risks  make  the  imcertainties  involved  in  contract  performance  too  great  to 
permit  costs  to  be  estimated  with  sufficient  accuracy  to  use  a  fixed-price  contract.  Offerors  were 
required  to  provide  a  cost/price  proposal  for  FY  00  and  01  requirements  as  part  of  their  proposal. 

The  contract  contains  a  six-year  period  for  issuance  of  DOs.  The  use  of  options  is  not 
considered  to  be  in  the  best  interest  of  the  Government.  The  technical  baseline  for  the  follow-on 
production  units  and  the  unique  trainer  requirements  associated  with  these  simulators  will  not  be 
accurately  known  until  the  Government  is  ready  to  issue  the  requisite  DOs  for  follow-on  units. 
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The  minimiim  award  expected  was  the  initial  procurement  of  training  system  analysis  and 
definition  to  be  acquired  under  the  first  DO,  estimated  to  cost  $6  million.  This  award  was  made 
simultaneously  with  the  IDIQ  award.  The  ceiling  amount  for  this  program  is  $300  million. 

Fimds  for  other  than  the  stated  minimum  quantity  are  to  be  obligated  at  the  time  of  individual 
DO  issuance,  during  FY  00  through  FY  05. 

The  Government  stated  upfront  that  it  intended  to  evaluate  proposals  and  award  a 
contract  without  discussions  with  the  offerors,  reserving  the  right' to  conduct  discussions  and 
request  proposal  revisions  if  it  was  determined  necessary. 

5.  Challenges  and  Advantages  of  Strategy 

Providing  a  budget  to  the  contractor  for  Government  work-years  in  an  EVMS.  This  task, 
necessary  to  the  full  implementation  of  the  IPT,  illustrates  the  difficulty  inherent  in  accoimting 
for  and  allocating  funds  budgeted  to  a  program.  The  program  office  recognized  that  the  best 
answer  is  to  have  Government  employees  working  in  IPTs.  Providing  oversight  and  employee 
control  through  compensation  is  complicated  with  this  arrangement. 

The  positive  side  to  the  use  of  Government  labor  is  that  it  serves  as  a  substitute  for 
competition.  The  existence  of  Government  labs  and  workers  willing  to  undertake  development 
efforts  of  this  sort,  regardless  of  ability  carries  weight  with  industry  similar  to  that  of  another 
competitor.  The  option  to  terminate  at  any  point  (coincident  with  the  end  of  a  delivery  order) 
also  compensates  for  awarding  the  IDIQ  contract  to  a  single  contractor.  Maintaining  competition 
through  multiple  awards  of  the  first  delivery  order  is  another  method,  if  not  overly  constrained 
by  costs,  to  maintain  competitive  pressure  to  perform. 

Establishing  joint  Government  and  industry  IPTs  involved  a  significant  amount  of  what  is 
commonly  referred  to  in  teamwork  literature  as  “forming,  storming  and  norming.”  This  chaotic 
period  where  the  team  resolved  initial  issues  as  to  how  best  to  effectively  accomplish  tasks 
assigned  resulted  in  the  perception  that  the  program  office  was  not  making  appropriate  progress. 
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Effort  was  required  to  convince  the  team,  the  program  office  and  senior  management  that  the 
program  was  not  “flailing”  and  making  progress  towards  achieving  the  objectives  of  the 
program.  This  effort  required  identification,  communication  and  understanding  of  the 
Government’s  role,  as  well  as  an  objective  assessment  of  the  Government’s  capability. 

Keeping  two  contractors  in  the  design  phase  meant  worrying  more  about  being  fair.  The 
decision  to  award  the  first  delivery  order  to  a  single  contractor  was  bounded  by  two  factors.  The 
first  was  the  cost  of  the  second  contract.  ‘  The  second  was  the  perception  that  the  real  cost  of 
having  two  contractors  performing  training  systems  analysis  and  design  outweighed  the  marginal 
benefit  of  possibly  integrating  another  better  answer  into  the  solution.  The  program  office  felt 
that  these  costs  would  be  measured  as  much  in  dollars  as  in  oversight  and  added  protest 
avoidance  type  behavior  on  their  part. 

Oral  presentations  made  on  two  of  four  evaluation  criteria  resulted  in  de  facto  elevation 
of  these  two  factors.  Two  of  four  equally  weighted  evaluation  criteria  were  program 
management  and  requirements  analysis.  These  two  factors  were  included  in  oral  presentations. 
This  resulted  in  these  two  factors  becoming  the  biggest  discriminators  between  contractors 
during  source  selection. 

Advantages  achieved  through  the  use  of  an  EDIQ  contract  include  flexibility  in  defining 
the  scope  of  work  required  for  follow-on  pattern  and  production  units,  assurance  of  technical 
compliance  and  affordability.  Utilization  of  IDIQ  contract  type  with  incremental  commitments 
with  contractors  also  facilitated  appropriate  contract  type  selection  and  adjustable  types  and 
quantities  of  simulators  acquired. 

Organizational  inertia  prevented  full  tailoring  of  acquisition  strategy.  The  program  office 
desired  to  tailor  their  program  to  most  efficiently  meet  their  objectives.  The  program  office  was 
open  to  ideas  as  to  how  best  to  streamline  and  tailor  their  program.  Shepherding  a  standard 
acquisition  approach  through  approving  levels  at  NAVAIR  took  18  months.  Operating  under 
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time  constraints  mostly  derived  from  funding  timelines  precluded  the  program  office  from 
bucking  the  system  to  further  tailor  their  approach. 

E.  JOINT  DIRECT  ATTACK  MUNITION  (JDAM) 

The  JDAM  program  was  started  after  operation  Desert  Storm  in  1991.  During  operation 
Desert  Storm  a  need  was  recognized  for  more  accurately  delivering  the  thousands  of  unguided 
munitions  in  military  stockpiles.  The  Government  requirement  was  stated  as  a  need  for 
increased  accuracy  munitions  usable  in  all  possible  weather  conditions  from  a  wide  variety  of 
aircraft.  The  vision  was  to  convert  gravity  bombs  into  precision  munitions  with  strap  on 
components.  The  collection  of  strap  on  components  to  convert  one  bomb  is  called  a  tail  kit.  (Ref 
22) 


Separate  Navy  and  Air  Force  programs  were  working  similar  programs  independently. 

In  1991,  these  programs  were  merged  to  form  JDAM.  The  Air  Force  was  appointed  lead 
program  manager.  Initially,  the  DoD  plarmed  to  purchase  40,000  tail  kits. 

JDAM  was  designated  a  Defense  Acquisition  Pilot  Program  (DAPP)  in  accordance  with 
the  Federal  Acquisition  Streamlining  Act  of  1994.  This  designation  authorized  relief  from 
numerous  DoD  regulatory  requirements.  In  the  case  of  JDAM,  25  FAR  and  25  DFARS  waivers 
were  granted.  The  most  significant  waiver  was  authority  to  use  commercial  practices  (Ref  23). 

DAPP  designation  spurred  program  management  to  utilize  new  approaches  to  conducting 
an  acquisition.  Waivers  were  not  needed  for  all  of  the  approaches  used,  however,  many 
proposed  approaches  were  outside  of  practices  normally  encountered  in  a  traditional  Defense 
acquisition.  Examples  of  different  approaches  incorporated  into  the  JDAM  program  include: 
(Ref  23) 


•  Use  of  a  rolling  down  select  process  emphasizing  continuous  information  exchange 
between  the  Government  and  industry  partners 
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•  Streamlined  oversight  of  the  program  and  contractors 

•  Use  of  a  commercial-like  warranty  covering  a  20-year  period 

•  A  long  term  Govemment/contractor  relationship  built  on  trust  with  aggressive  use  of 
IPTs  that  included  Government  personnel  on  industry  teams  at  contractor  sites 

•  Program  and  contractor  management  at  the  system  specification  level 

•  Use  of  contractor  formats  for  most  data  submittals,  program  reviews,  design  reviews 
and  earned  value  reporting 

•  Use  of  long-term  prime/subcontractor  relationships 

1 .  Inten  ded  Strategy 

Two  Engineering  and  Manufacturing  Development  (EMD)  phases  were  used  in 
sequence.  Phase  I  EMD  objectives  were  to  reduce  manufacturing  risk  and  projected  Average 
Unit  Production  Price  (AUPP).  Phase  I  was  scheduled  to  take  18  months  to  complete.  Phase  II 
EMD  objectives  were  to  complete  development  and  operational  testing.  The  phase  II  EMD 
contract  was  a  cost-plus  award  fee  contract.  This  contract  included  economic  order  quantity 
options  for  the  first  two  production  lots.  Production  price  commitment  curves  for  minimum  and 
economic  order  quantities  were  included  for  lots  three  through  five. 

2.  Barriers  to  Use  of  Intended  Strategy 

Military  regulations  and  specifications  typical  to  most  defense  acquisition  programs 
inhibited  innovations. 

The  JDAM  program  began  using  a  traditional  acquisition  approach.  As  discussed  in  the 
Arsenal  Ship  case,  this  approach  inhibited  soimd  business  evaluation  of  contractor  performance. 
An  excessively  formal  process  with  starts  and  stops  pimctuated  by  “over  the  wall” 
communication  between  buyer  and  seller  mark  this  traditional  process.  Government  feedback  to 
contractors  can  be  met  with  uncertainty  regarding  the  appropriate  response.  One  choice  of 
response  is  to  ignore  the  feedback.  (Ref  21,22,25) 
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While  iinder  consideration  for  designation  as  a  Defense  Acquisition  Pilot  Project,  there 
was  resistance  to  using  commercial  item  provisions  proposed  for  the  FAR  in  the  JDAM  program. 
The  Deputy  Under  Secretary  of  Defense  for  Acquisition  Reform  was  responsible  for 
recommending  pilot  projects.  The  direction  the  Acquisition  Reform  office  was  taking  with 
Congress  was  to  use  programs  procuring  “semi-commercial”  products  as  pilots.  In  this  case 
defining  semi-commercial  products  as  items  normally  purchased  by  commercial  firms.  JDAM 
was  procuring  items  with  military  applications  (bomb  conversion  kits).  By  limiting  pilot  projects 
to  semi-commercial  items  program  managers  of  military  programs  could  likely  conclude  that  this 
commercial  approach  would  not  be  applicable  to  weapons  systems  that  are  not  semi-commercial. 
Success  in  the  JDAM  program  using  a  commercial  approach  was  proposed  as  a  means  to  dispel 
this  notion.  (Ref  22) 

At  the  beginning  of  the  JDAM  program,  incentives  were  not  in  place  to  properly  drive 
cost  reduction.  The  cost-based  approach  used  had  the  contractor  compiling  reams  of  data  to 
show  the  cost  of  an  item.  If  an  item  cost  $1,000  to  build  and  10%  profit  was  provided  for,  the 
Government  would  pay  a  total  of  $1,100  for  the  item.  If  the  contractor  subsequently  reduced  the 
build  cost  to  $900,  the  profit  would  be  $90  with  a  total  cost  to  the  Government  of  $990.  The 
contractor  reduced  profit  with  increased  efficiency.  (Ref  22) 

Government  experts  lacked  currency  of  their  authority  relative  to  technology.  IPTs  that 
proposed  alternate  solutions  for  processes  experienced  this.  In  some  cases.  Government  experts 
who  were  technologically  current  several  generations  of  technology  ago  had  one  perception  on  a 
specific  issue.  When  confronted  with  alternate  processes  that  did  not  mesh  with  dated 
perceptions,  responses  included  increased  resistance  to  the  alternate  process.  Organizational 
inertia  manifested  itself  in  this  manner  with  potential  to  cause  cost  overruns.  (Ref  22) 
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3. 


Expected  Benefits  from  Use  of  Intended  Strategy 


A  sensible  evaluation  of  contractor  performance  was  an  expected  benefit  in  the  JDAM 
strategy.  Structuring  contractor  feedback  in  the  rolling  down  select  would  enhance 
communication  between  the  Government  and  industry.  (Ref  25) 

The  rolling  down  select  process  forced  a  match  between  Government  feedback  to 
contractors  and  down  select  evaluation  criteria.  This  had  two  important  outcomes;  First,  the 
Government  had  to  carefully  consider  what  was  most  important  in  the  evaluation  for  down 
selection.  Second,  before  the  evaluation  for  the  actual  down  select  took  place,  the  Government 
had  practiced  evaluating  contractors  against  the  evaluation  criteria.  The  expected  benefit  is  an 
increased  likelihood  of  selecting  the  best  value  contractor  as  opposed  to  picking  the  contractor 
with  the  most  polished  proposal.  (Ref  25) 

DAPP  Designation  permitted  JDAM  to  purchase  military  products  as  if  they  were 
commercial  items.  Regulations,  specifications  and  reports  required  in  traditional  acquisition 
programs  were  minimized.  The  expected  benefit  is  cost  reduction.  (Ref  24)  Metrics  that  could 
be  traced  to  cost,  schedule  and  performance  were  used  and  included:  (Ref  23) 

•  Program  cost  including  Average  Unit  Procurement  Price  (AUPP) 

•  Cost  estimate  change  rationale 

•  Program  office  staffing 

•  Operational  performance 

•  Unit  Cost  Comparison  (JSOW/JDAM  Guidance  Control  Units) 

•  Program  funding  stability 

•  Would  cost  analyses 

•  Regulatory/statutory  relief 
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The  expected  benefit  of  using  these  metrics  was  reduced  program  cost  compared  to  the 
baseline  estimate.  Additional  expected  benefits  included  paperwork  reduction  and  streamlined 
oversight. 

4.  Execution  of  Strategy 

JDAM  started  out  as  a  traditional  program.  Prior  to  DAPP  designation  it  was  conducted 
as  a  traditional  acquisition  program.  This  included  a  portion  of  the'  RFP  cycle.  (Ref  22) 

JDAM  implemented  CAIV  in  its  phase  I  selection  criteria  by  assigning  the  most  weight 
to  affordability  (Ref  24).  The  DoD  Acquisition  Reform  Office  was  promoting  CAIV.  The 
JDAM  program  office  interpretation  of  CAIV  was  that  as  long  as  their  five  key  criteria  were  met, 
everything  else  was  fair  game  for  changing  to  lower  cost.  In  this  manner  the  contractor  could 
manage  its  own  costs.  EPTs  worked  with  suppliers  to  analyze  and  eliminate  or  reduce  cost 
drivers  in  each  system  component.  (Ref  22) 

Prior  to  phase  I EMD  the  program  office  surveyed  best  practices  in  commercial 
companies.  Specifically  targeted  were  processes  and  practices  between  buyers  and  key 
suppliers.  Findings  of  the  survey  were:  (Ref  24) 

•  Buyer-seller  relationships  tended  to  be  collaborative  rather  than  adversarial 

•  Buyer-seller  relationships  were  often  long-term  and  exclusive 

•  Contract  negotiations  normally  focused  on  the  price  charged  to  the  buyer  rather  than 
the  cost  incurred  by  the  supplier 

•  Contract  documents  were  usually  short  and  simple 

•  Contract  requirements  generally  did  not  change  once  they  were  established 

•  Buyers  seldom  tried  to  dictate  to  suppliers  how  they  should  do  their  jobs 

•  Past  performance  played  a  big  part  in  the  selection  of  suppliers 
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These  benchmarks  were  implemented  as  the  following  program  goals:  (Ref  22) 


•  Government/ supplier  integrated  product  teams  (IPT s) 

•  Performance  based,  head-to-head  competition 

•  Rolling  down  select  (three  report  cards  during  competitive  phase) 

•  Allowing  the  contractor  control  over  the  technical  data  package 

•  Requiring  a  contractor-supplied  warranty 

•  Minimal  paperwork  and  limited,  streamlined  oversight 

•  Negotiations  based  on  supplier  price,  not  cost 

•  Primary  award  criteria  based  on  past  performance  and  best  value 

•  Allowing  trade-offs  of  price  for  performance  criteria  (except  for  a  few  live-or-die 
criteria) 

•  Firm  fixed-price  production  contract 

•  Use  of  commercial  products 

In  April  1994,  two  out  of  five  competitors  were  awarded  phase  I  HMD  contracts.  Martin 
Marietta  Corporation  and  McDormell  Douglas  Aerospace  were  the  prime  contractors.  (Ref  24) 

JDAM  was  designated  a  DAPP  1 1  days  after  phase  I EMD  contract  award  (Ref  24).  In 
August  of  1994,  the  program  office  encouraged  the  contractors  to  rewrite  their  initial  100-plus 
page  SOW.  The  desired  product  was  a  two-page  SOO.  The  contractors  deleted  the  majority  of 
the  paper  deliverables,  military  standards  and  specifications  (Ref  22).  In  Fall  1994,  USD  (A, 
Ti&L)  accelerated  the  program  by  approximately  1 8  months  to  field  improved  weapons  as  soon 
as  possible. 

Emulating  commercial  practices,  program  managers  crafted  a  framework  of  incentives 
for  contractors  to  meet  performance  and  price  goals.  If  the  contractor  kept  the  price  below  that  it 
had  committed  to,  the  following  rewards  applied:  (Ref  24) 

•  It  did  not  need  to  submit  cost  data  justifying  its  price  or  technical  proposals 
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•  It  had  complete  control  over  the  system  configuration  so  long  as  all  specifications 
were  met 

•  It  had  access  to  Government  assistance  in  reducing  costs  without  being  obliged  to 
pass  on  savings  in  the  form  of  lower  prices 

•  It  did  not  have  to  deal  with  direct  Government  oversight  of  its  plants  or  production 
processes 

•  It  did  not  have  to  worry  about  the  Government  seeking  a  second  source  that  would 
compete  for  the  business 

•  It  had  exclusive  responsibility  for  repairing  and  maintaining  the  product 

•  It  received  an  additional  fee  for  exceeding  the  accuracy  and  reliability  specifications 

The  opposing  group  of  incentives  was  used  to  improve  the  level  of  contractor  security 
and  profit  when  contrasted  to  traditional  acquisition  programs.  If  the  price  rose  above  that  it  had 
committed  to,  the  following  sanctions  applied:  (Ref  24) 

•  It  had  to  provide  detailed  cost  and  pricing  data  to  the  Government 

•  It  had  to  provide  detailed  technical  data  to  the  Government  suitable  for  preparing  a 
reprocurement 

•  It  had  to  develop  an  alternate  production  source  at  no  cost  to  the  Government 

•  It  lost  configuration  control  over  the  system,  which  reverted  to  the  Government 

•  It  lost  any  fees  it  might  otherwise  have  collected  as  a  reward  for  surpassing  accuracy 
and  reliability  specifications 

•  It  potentially  lost  the  responsibility  for  repairing  and  maintaining  the  product 

•  It  potentially  lost  the  fi-eedom  firom  in-plant  and  in-process  oversight  by  Government 
inspectors 

The  missile  development  industry  has  an  organizational  history  of  innovation  and 
teamwork.  A  large  number  of  the  engineers  that  made  up  industry  program  offices  were 
schooled  in  this  teamwork-oriented  environment.  McDonnell  Douglas  was  fimctionally 
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organized.  The  Government  decision  to  use  BPTs  drove  McDonnell  Douglas  to  reorganize  in  a 
way  that  facilitated  the  use  of  IPTs.  (Ref  22) 

Through  their  actions  and  words,  the  program  office  initially  communicated  to  the 
contractors  that  the  philosophy  was  "perform  to  plan."  During  phase  I EMD  a  shift  in 
philosophy  emerged,  and  AUPP  for  the  first  two  productions  lots  became  the  ultimate  evaluation 
criteria.  At  this  point  in  the  acquisition,  control  of  the  Technical  Data  Package  (TDP)  was  given 
to  the  contractors  to  assist  them  in  their  cost  control  efforts.  Meeting  key  performance 
requirements  provided  the  framework  from  within  which  the  TDP  could  be  modified.  (Ref  22) 

The  award  fee  and  down  select  criteria  had  evolved  from  a  traditional  source  selection  to 
a  rolling  down  select  (Ref  22).  Contractors  received  formal  feedback  three  times  during  phase  I 
EMD.  Color-coded  grades  based  on  down  select  evaluation  criteria  were  given  to  the  contractors 
in  open  discussions.  Grades  were  assigned  based  on  Government  perception  of  how  well  output 
met  expectations  vice  a  relative  standing  with  competitors.  The  feedback  given  was  factored 
into  the  final  selection  decision.  The  desired  outcome  of  this  process  was  for  the  SSA  to  receive 
self-reported  progress  from  the  contractors  and  to  provide  opportunities  for  contractors  to 
improve  their  performance  (Ref  23). 

The  initial  design  phase  (phase  I  EMD)  was  successful.  The  next  phase,  phase  II  EMD, 
was  more  difficult.  Fabrication  and  extensive  testing  were  included  (Ref  22).  The  phase  II  EMD 
award  decision  was  based  on  the  AUPP  and  meeting  the  key  performance  criteria  (Ref  22). 
McDonnell  Douglas  was  selected  as  the  phase  II  EMD  contractor  in  October  1995  (Ref  24). 

JDAM  production  rate  is  expected  to  exceed  1000  kits  per  month  during  production.  The 
DoD  plans  to  buy  87,496  JDAMs  for  use  by  the  Air  Force,  Navy  and  Marine  Corps  over  a  ten- 
year  production  period.  Export  licenses  have  been  approved  and  significant  international  sales 
are  expected. 
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5.  Challenges  and  Advantages  of  Strategy 

Candid  discussions  with  contractors  and  well  rmderstood  evaluation  criteria  resulted  in 
reduced  costs  and  cycle  times.  Phase  I  and  II  contracts  were  awarded  with  30  percent  less  effort 
than  expected  and  bid  and  proposal  costs  were  reduced  by  50  percent  (Ref  22, 23).  The  unique 
structure  of  the  acquisition  facilitated  reduced  program  office  staffing  (Ref  23).  Contract 
administration  measured  in  hours  was  reduced  85  percent  to  date  (Ref  23).  AUPP  was  reduced 
fi'om  $40,000  to  $14,000.  R&D  costs  were  reduced  from  $380  to  $310  million.  '  The 
development  portion  of  the  program  was  shortened  by  1 6  months  to  a  total  of  30  months.  The 
cycle  time  of  the  entire  program  has  been  reduced  fi'om  15  to  10  years.  (Ref  22)  The  total  cost 
avoidance  predicted  for  JDAM  over  the  ten-year  production  cycle  is  $2.96  billion  (Ref  23). 

Increased  communications  minimized  the  potential  for  protest  from  eliminated 
competitors  (Ref  22).  Typically,  Government  response  to  contractor  proposals  follows  the 
formal  source  selection  process.  This  limited  response  to  proposals  is  recognized  as  an 
underlying  cause  of  award  protests.  The  Government’s  desire  to  avoid  the  risk  of  protests  often 
results  in  excessive  documentation  requirements.  This  was  also  avoided  (Ref  22) 

Maintaining  control  of  costs  in  the  bureaucracy  of  defense  acquisitions  was  a  challenge 
during  execution  of  the  JDAM  program.  When  the  JDAM  program  was  initiated,  CATV  was  a 
proposal  that  had  not  been  put  into  practice  (Ref 22, 24). 

Program  office  manning  was  a  challenge.  The  aspect  of  manning  that  presented  a 
challenge  was  filling  positions  with  change-minded  individuals  open  to  operating  with 
innovative  business  practices.  Managers  were  needed  that  believed  that  alternative  processes 
could  work  (Ref  24).  Corporate  Defense  acquisition  memory  includes  cases  of  managers  with 
insufficient  understanding  of  technical  and  other  issues  to  guide  programs  to  success.  A 
potential  result  of  this  circumstance  is  meeting  systems  requirements  with  either  prohibitive  costs 
or  difficult  to  manufacture  products.  Including  all  relevant  Government  personnel  while 
maintaining  teamwork  and  open  mindedness  is  the  challenge.  DAPP  designation  provided 
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opportunities  to  the  JDAM  program  that  required  overcoming  organizational  resistance  to 
change.  Difficulties  of  this  nature  were  overcome  in  the  JDAM  program  with  strong 
interpersonal  communication  at  all  levels  and  willingness  to  take  risks. 

Phase  I  successes  resulted  from  the  application  of  changed  behavior  patterns.  Personnel 
were  not  sufficiently  rewarded  for  these  new  behaviors.  Close  relationships  built  in  phase  I 
migrated  back  to  less  collaborative  forms.  The  program  office  was  under  pressure  to  achieve 
success  in  phase  II EMD  similar  to  that  seen  in  phase  I.  Workforce  incentives  have  not  been 
established  to  sustain  continued  similar  efforts.  (Ref  22) 

The  rolling  down  select  feedback  process  provided  stronger  incentive  for  contractors  to 
focus  their  efforts  than  either  incentives  or  award  fees  (Ref  25).  This  can  be  attributed  to 
heightened  contractor  interest  in  a  long-term  relationship  as  opposed  to  earning  award  fee  on  the 
instant  contract. 

The  likelihood  of  selecting  the  best  value  contractor  as  opposed  to  picking  the  contractor 
with  the  most  polished  proposal  was  increased.  The  rolling  down  select  process  forced  a  match 
between  Government  feedback  to  contractors  and  down  select  evaluation  criteria.  This  had  two 
important  outcomes.  First,  the  Government  had  to  carefully  consider  what  was  most  important 
in  the  evaluation  for  down  selection.  Second,  before  the  evaluation  for  the  actual  down  select 
took  place,  the  Government  had  practiced  evaluating  contractors  against  the  evaluation  criteria. 
While  significant  workload  was  required  to  use  this  process,  it  is  unlikely  that  the  workload  was 
more  than  monitoring  and  administering  an  award  fee.  (Ref  25) 

Contractor  performance  was  motivated  towards  appropriate  areas  through  the  feedback 
process  used.  DAPP  Designation  deregulated  the  JDAM  program.  Alternative  control 
mechanisms  were  required.  McDonnell  Douglas  coimected  rolling  down  select  grades  directly 
to  individual  compensation.  Bonuses  were  based  on  color  grades  received.  (Ref  22) 
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Using  Commercial  Off  The  Shelf  (COTS)  products  reduced  the  time  and  expense  of 
development.  Although  as  a  whole  militarily  unique,  over  80  percent  of  JDAM  components  are 
COTS.  Not  using  military  regulations  and  specifications  facilitated  this.  (Ref  24) 

Full  CATV  implementation  required  a  modified  approach.  The  following  principles  were 
critical  to  achieving  maximum  benefit  from  CATV :  (Ref  24) 

•  Realistic  acquisition  and  life  cycle  cost  objectives  had  to' be  stated  at  the  outset  and 
maintained 

•  Performance  requirements  and  schedule  goals  against  which  cost  was  being  traded  off 
had  to  be  clear  and  no  more  demanding  than  absolutely  necessary 

•  Military  specifications  had  to  be  minimized  so  they  did  not  interfere  with  using 
commercial  products  and  processes 

•  There  had  to  be  rigorous  metrics  and  methods  for  tracking  program  progress 

•  Contractors  had  to  be  motivated  to  adhere  to  their  price  and  performance 
commitments 

•  The  program  office  had  to  be  manned  with  personnel  who  possessed  experience  bases 
that  facilitated  their  ability  to  understand  the  CAfV  vision 

An  advantage  achieved  is  the  commercial  like  warranty.  The  JDAM  warranty  requires 
the  contractor  to  repair  or  replace  any  kit  that  fails  within  the  specified  shelf  life  (20  years)  and 
service  life  (5  years)  periods.  The  warranty  applies  to  all  production  lots.  On  the  first  four  lots,  a 
repaired  or  replacement  kit  must  be  shipped  within  one  business  day  after  official  notification 
that  a  failed  unit  is  being  returned  by  the  Government.  40  business  days  are  allotted  for 
subsequent  production  lots,  with  the  contractor  paying  all  replacement  and  shipping  costs.  (Ref 
24) 


A  challenge  facing  a  DAPP  and  other  pilot  projects  is  a  lack  of  guidance  on  how  to 
proceed  (Ref  22).  JDAM  program  managers  sought  first  to  understand  the  intent  of  the  pilot. 
JDAM  program  managers  believed  the  intent  was  to  conduct  a  Government  acquisition  in  the 
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manner  of  a  commercial  enterprise.  A  survey  identified  best  practices  that  were  then  used  as 
program  goals.  (Ref  22) 

A  challenge  facing  the  JDAM  program  was  workforce  and  industry  partner  willingness  to 
change  (Ref  22). 

Subcontractors  supplied  over  80  percent  of  the  components  that  made  up  JDAM. 
Involving  these  suppliers  in  the  CAIV  process  was  a  challenge.  Facilitating  open  and 
collaborative  communication  overcame  this  challenge.  Non-disclosure  agreements  facilitated 
openness  in  communications.  Program  management  took  the  first  step  by  sharing  cost  goals  and 
tracking  charts  which  set  a  precedence  for  open  communications.  (Ref  22) 

F.  CHAPTER  SUMMARY 

The  four  programs  profiled  in  this  Chapter  share  a  common  theme  of  innovation  and  the 
attempt  of  some  form  of  phasing  in  source  selection.  The  next  Chapter  will  take  an  in-depth  look 
at  source  selection  policy  and  procedures  at  the  NAVAIR. 

Further  analysis  of  the  points  noted  in  the  case  studies  in  this  Chapter  will  be 
accomplished  in  Chapter  V. 
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V.  NAVAL  AIR  SYSTEMS  COMMAND 


A.  INTRODUCTION 

NAVAIR  is  one  of  three  main  hardware  systems  commands  utilized  by  the  Navy  to 
procure  weapon  systems. 

NAVAIR  is  organized  using  a  team  concept:  Six  organizations  make  up  the  team:  '  ’ 
NAVAIR;  Naval  Inventory  Control  Point  (NAVICP);  Program  Executive  Office,  Air  Anti- 
Submarine  Warfare,  Assault,  and  Special  Mission  Programs  PEO  (A);  Program  Executive 
Office,  Tactical  Aircraft  Programs  PEO  (T);  Program  Executive  Office,  Strike  Weapons  and 
Unmanned  Aviation  PEO  (W);  and  Program  Executive  Office,  Joint  Strike  Fighter  PEO  (JSF). 

The  Naval  Aviation  Systems  Team  (TEAM)  considers  itself  to  be  a  partner  with  industry. 
The  TEAM  develops,  acquires,  and  supports  naval  aeronautical  and  related  technology  systems 
for  use  by  operating  forces.  Their  stated  objective  is  to  provide  technology  solutions  that 
increase  warfighter  capabilities  and  effectiveness  in  future  warfighting  scenarios. 

Products  and  services  provided  to  customers  include:  Navy  and  Marine  Corps  aircraft 
systems;  air-launched  weapons  systems  and  subsystems;  airborne  electronics  systems;  air- 
laimched  underwater  sound  systems;  airborne  pyrotechnics;  astronautics  and  spacecraft  systems; 
airborne  mine  coimtermeasures  equipment  (except  for  explosives,  explosive  components,  and 
fusing);  aeronautical  drones  and  towed  target  systems,  including  related  ground  control 
equipment  and  launch  and  control  aircraft;  meteorological  equipment;  overhaul  and  modification 
of  all  Naval  aircraft/engines;  operation  and  maintenance  of  weapons  training  ranges.  Total  life 
cycle  support  is  provided  and  includes:  research,  design,  development,  and  engineering; 
acquisition;  test  and  evaluation;  training  facilities  and  equipment;  repair  and  modification;  and 
in-service  engineering  and  logistics  support. 
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Establishing  a  source  selection  strategy  at  NAVAIR  for  major  acquisitions  is 
accomplished  jointly  by  either  program  managers  or  contracting  officers  working  with  source 
selection  resident  experts.  These  resident  experts  operate  out  of  an  office  physically  separated 
from  program  management  and  contracting  personnel.  This  source  selection  body  is  referred  to 
by  their  organizational  code,  NAVAIR  4.10. 

A  Source  Selection  Policy  Board  (SSPB)  meets  periodically  to  discuss  and  effect 
improvements  to  source  selection.  Consistency  in  source  selection  procedures  is  their  official 
objective.  The  SSPB  is  chaired  by  the  Deputy  Commander  for  Acquisition  and  Operations.  The 
board  is  comprised  of  senior  representatives  from  the  Assistant  Commander  for  Contracts,  the 
Assistant  Commander  for  Research  and  Engineering,  and  the  Office  of  Legal  Counsel  (Ref  26). 
Any  changes  to  source  selection  policy  or  procedures  have  to  be  approved  by  the  SSPB  (Ref  28). 
Execution  of  the  source  selection  policy  board  is  seen  as  an  ad  hoc  approach  by  members  of  the 
NAVAIR  contracting  competency  (Ref  30). 

B,  CURRENT  SOURCE  SELECTION  STRATEGIES 

Source  selections  are  conducted  utilizing  one  of  three  general  methods;  formal,  tailored 
and  Competitive  Award  Panel  (CAP)  (Ref  26). 

Dollar  value  thresholds  determine  which  method  to  use.  Formal  source  selection  is  used 
for  ACAT I  and  II  programs  (major  systems).  Tailored  or  CAP  are  used  for  acquisitions  that  are 
not  major  systems  but  are  expected  to  obligate  more  than  $30  million  in  RDT&E  funds  and  $150 
million  in  procurement  funds.  CAP  is  used  for  acquisitions  expected  to  obligate  less  than  $30 
million  in  RDT&E  funds  and  $  1 50  million  in  procurement  funds  (Ref  26).  The  decision  of 
which  process  to  utilize  is  made  at  the  program  level  IPT  (Ref  29). 

The  formal  source  selection  procedure  lays  out  an  evaluation  group  structure.  This  group 
structure  is  managed  by  NAVAIR  4.10  and  evaluates  proposals  and  recommends  a  source  for 
contract  award  (Ref  26).  Approximately  five  percent  of  competitions  utilize  this  process,  or 
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somewhere  between  four  to  twelve  source  selections  per  year  (Ref  29).  The  majority  of 
contracts  awarded  utilizing  this  process  are  cost  plus  types  (Ref  29). 

Tailored  source  selections  use  a  streamlined  version  of  the  formal  evaluation  group 
structure.  This  structure  is  managed  by  NAVAIR  4.10  (Ref 26, 28).  Visibility  of  the  acquisition 
and  the  amount  of  program  risk  are  factors  considered  when  choosing  between  a  formal  or  a 
tailored  source  selection  (Ref  26). 

The  CAP  process  is  a  further  streamlined  process.  NAVAIR  4.10  does  not  manage  this 
process.  Generally,  the  SSA  is  supplied  by  the  program  office.  The  preference  is  to  appoint  a 
SSA  with  experience  in  competitive  negotiated  acquisitions,  and  to  provide  the  SSA  with 
technical  and  other  support  (Ref  26).  SSA  is  meant  to  be  delegated  as  low  as  possible  within  the 
program.  The  delegation  level  is  outlined  in  the  acquisition  strategy,  which  is  staffed  by 
NAVAIR  4.10  (Ref  26, 28).  The  PCO  generally  chairs  the  CAP  and  is  the  SSA.  The  SSA  could 
also  be  the  program  manager. 

PTs  are  used  in  all  acquisitions.  The  lead  IPT  determines  source  selection  evaluation 
team  membership. 

Early  plaiming  and  preparation  is  emphasized  in  the  source  selection  process. 
Noteworthy  in  instmctional  guidance  on  what  constitutes  early  planning  is  the  finalization  of  all 
program  requirements  (Ref  26). 

Source  selections  generally  follow  these  steps:  (Ref  26) 

•  Procurement  ff  T  agreements 

•  Acquisition  strategy 

•  Synopsis 

•  Draft  solicitation 

•  Pre-solicitation  conference 
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•  Source  selection  plan 

•  Solicitation 

•  Pre-proposal  conference 

•  Award  on  initial  offers  or  determination  of  competitive  range 

•  Site  surveys 

•  Pre-negotiation  business  clearance 

•  Discussions 

•  Request  for  final  proposal  revisions 

•  Evaluation  documentation 

•  Selecting  a  source 

•  Post-business  clearance 

•  Contract  award 

•  Debriefing 

Fair  and  impartial  evaluation  of  proposals  is  the  expected  outcome  of  having  and  using  a 
systematic  and  comprehensive  source  selection  process.  The  best  value  contractor  is  sought. 
Resolution  of  offeror  issues  pre-award  and  considering  all  relevant  factors  while  evaluating  for 
best  value  is  intended  to  avoid  protests.  (Ref  27)  Protest  avoidance  and  a  focus  on  the  ability  to 
award  without  discussions  permeates  source  selection  (Ref  29). 

Evaluating  proposals  with  point  systems  is  not  done.  Normally,  adjectival  rating  systems 
are  used.  Point  systems  allow  the  possibility  of  masking  critical  deficiencies.  Adjectival 
systems  provide  mechanisms  to  highlight  discriminating  factors  between  proposals  and  can 
better  respond  to  xmanticipated  proposal  content  (Ref  27). 

Past  performance  is  used  as  a  separate  evaluation  factor.  The  intent  is  to  weed  out 
offerors  more  skilled  in  proposal  writing  than  performance  of  work.  Using  PPI  provided  by 
parties  other  than  the  offeror  is  stressed.  (Ref  27) 
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Cost  segregation  is  emphasized  in  proposal  evaluation.  Proposal  costs  are  generally  not 
made  available  to  technical,  past  performance  or  other  evaluation  teams.  This  is  intended  to 
prevent  cost  from  influencing  SSEB  evaluations  in  specific  areas.  (Ref  27) 

Senior  management  at  NAVAIR  tends  to  favor  the  formal  source  selection  process  (Ref 
28, 29, 30).  This  bolsters  the  entrenchment  of  rigid  application  of  procedure  in  NAVAIR  4.10. 
Highly  technical,  complex  and  high  visibility  acquisitions  are  seen  as  needing  an  engineering 
focus  (Ref  28).  NAVAIR  4.10  was  established  in  an  engineering  discipline  in  the  1950s  after  a 
sustained  protest  due  to  mistakes  in  the  technical  evaluation.  These  mistakes  were  made  as  a 
result  of  a  lack  of  technical  imderstanding  on  the  part  of  the  PCO  and  SSA  (Ref  28). 

The  placement  of  NAVAIR  4.10  forms  the  underpinning  of  a  philosophical  argument 
concerning  organizational  placement  of  the  source  selection  experts  (Ref  28, 30).  One  train  of 
thought  is  to  have  a  balance  of  power  in  source  selection  between  engineering  and  business 
elements.  NAVAIR  4.10  is  an  engineering  discipline,  with  membership  made  up  predominantly 
of  personnel  with  backgrounds  in  engineering  disciplines.  The  opposing  train  of  thought  is  that 
source  selection  should  have  a  more  business  or  contract  oriented  position,  staffed  with 
persoimel  having  business  backgrounds  and  organizationally  positioned  outside  of  the 
engineering  field  either  in  contracts  or  a  financial  area  (Ref  30).  NAVAIR  4.10  as  currently 
structured  serves  as  the  “honest  broker”  in  the  balance  of  power  (Ref  28).  NAVAIR  4.10  leaders 
interpret  their  main  value  to  be  a  full  imderstanding  of  the  source  selection  process  and  the 
ability  to  refine  it  to  a  rigid  structured  process  (Ref  28). 

C.  ORGANIZATIONAL  EFFORTS  TAKEN  TO  IMPROVE  SOURCE  SELECTION 

1.  Source  Selection  Policy  Board 

Differences  in  program  relationships  with  NAVAIR  4.10  are  spread  along  a  spectrum 
from  adversarial  to  working  together  (Ref  28).  Differences  also  exist  between  programs  in 
execution  of  intended  strategies  and  regulations  (Ref  28, 29).  NAVAIR  4.10  looks  for  policy 
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groups  embedded  in  programs  to  act  as  referees  between  programs  and  NAVAIR  4.10  and  to 
maintain  consistency  in  application  of  source  selection  methodology  and  procedural  execution 
(Ref  28).  The  SSPB  is  the  forum  for  minimizing  differences  between  programs  and  acts  as 
referee.  The  SSPB  has  been  in  place  for  several  years.  In  the  past,  the  SSPB  has  revised  the 
NAVAIR  source  selection  instruction  to  reflect  changes  brought  about  by  the  Federal 
Acquisition  Streamlining  Act  (FASA),  the  Federal  Acquisition  Reform  Act  (FARA),  and  the 
FAR  Part  15  rewrite  (Ref  31).  The  most  recent  revision  of  the  NAVAIR  source  selection 
instruction  is  dated  22  July  1999  (Ref  31).  Currently,  the  SSPB  meets  monthly  to  discuss  current 
issues  in  source  selection. 

2.  Use  of  Past  Performance  as  a  Source  Selection  Evaluation  Factor 

NAVAIR  has  undertaken  steps  to  more  aggressively  use  PPI  to  evaluate  potential  sources 
when  evaluating  offerors  for  contract  award.  PPI  has  been  used  as  a  factor  by  NAVAIR  in 
source  selections  for  approximately  ten  years.  Difficulties  in  using  PPI  as  a  true  discriminator 
resulted  in  organizational  resistance  to  using  PPI  until  approximately  1998.  Specific  difficulties 
included  having  personnel  with  sufficient  past  performance  evaluation  experience,  and  the  lack 
of  an  accessible  database  of  information  on  contractor  performance.  The  FAR  policy  shift 
elevating  the  use  of  PPI  in  source  selections  and  the  subsequent  DoD  requirement  for  annual 
assessment  reports  on  contractor  performance  facilitated  overcoming  the  second  difficulty. 
NAVAIR  was  a  leader  in  documenting  PPI  in  the  Contractor  Performance  Assessment  Reporting 
System  (CPARS). 

CPARS  is  a  web-based  tool  for  data  basing  PPI.  A  CPAR  contains  an  assessment  of  a 
contractor's  performance.  A  CPAR  can  be  positive,  negative,  or  both.  It  reflects  performance 
during  a  contract  at  a  specific  time.  The  assessments  are  supposed  to  be  based  on  objective  facts 
and  supported  by  data  like  cost  performance  reports,  customer  comments,  quality  reviews, 
financial  assessments,  management  reviews,  functional  performance  evaluations,  and  earned 
contract  incentives. 
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Prior  to  implementing  CPARS,  PPI  was  predominantly  collected  through  the  use  of 
surveys.  Surveys  have  proven  to  be  not  very  effective  in  collecting  accurate  and  useful  PPI.  The 
mandate  for  all  services  to  perform  annual  PPI  assessments  has  generated  an  improved  and  more 
accessible  database  of  PPI. 

The  difficulty  of  having  sufficient  experienced  personnel  to  evaluate  contractor  past 
performance  has  yet  to  be  overcome.  Contrasted  to  technical  evaluation,  past  performance  is  not 
as  black  and  white.  It  is  more  difficult  to  arrive  at  an  absolute  answer  whem faced  with  ranking 
one  firms  past  performance  as  better  or  worse  than  another.  This  “squishiness”  of  PPI,  when 
combined  with  a  risk-averse  mindset  elevates  the  level  of  concern  for  acceptability  of  risk  in 
terms  of  the  potential  for  protests.  The  structure  of  program  teams  and  assignment  of  program 
personnel  to  evaluate  PPI  in  source  selections  has  resulted  in  two  difficulties:  first,  difficulty 
developing  personnel  with  solid  PPI  evaluation  experience  and  second,  consistently  assigning 
these  personnel  to  source  selection  evaluation  teams. 

Currently,  NAVAIR  source  selections  weigh  PPI  equal  to  other  source  selection 
evaluation  factors.  If  not  the  most  important  source  selection  factor,  past  performance  will  be 
equal  to  the  other  factors. 

3.  Price-Based  Acquisition  Tool  Identification  and  Development 

The  contracting  competency  (NAVAIR  2.0)  has  identified  a  small  group  of  PBA  subject 
matter  experts.  These  subject  matter  experts  are  listed  in  an  online  guide  that  directs  contracting 
and  program  persoimel  to  those  with  specific  experience  in  topical  areas.  The  identification  and 
use  of  this  group  is  intended  to  promote  the  use  of  PBA  tools.  The  expanded  use  of  PBA  tools  is 
also  intended  as  a  means  to  further  evaluate  proposed  methods  and  to  take  advantage  of 
successful  strategies. 

Group  members  are  contracting  officers  with  the  Harpoon  and  Tomahawk  weapon 
systems.  Both  the  Harpoon  and  Tomahawk  programs  have  utilized  PBA.  In  both  cases. 
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awarding  contracts  using  price  versus  cost  data  was  deemed  possible  due  to  the  history  of  the 
programs.  In  both  cases,  the  items  being  acquired  had  been  in  production  for  a  number  of  years, 
with  significant  cost  history  developed  and  documented.  Both  programs  have  stable 
requirements  and  configurations. 

D.  CHAPTER  SUMMARY 

Elements  are  in  place  to  take' advantage  of  organizational  know-how  in  source  selection. 
A  sound  structure  consisting  of  NAVAIR  4.10,  the  SSPB  and  the  price-based  subject  matter 
experts  is  laid  out.  Although  NAVAIR  has  taken  steps  to  enhance  the  use  of  alternative  source 
selection  strategies,  such  as  PBA,  the  current  infrastructure  appears  to  support  the  status  quo  of 
traditional  source  selection  strategies  for  major  systems. 

The  risk-averse  tendency  of  senior  management  combined  with  the  rigidity  of  policy 
application  by  NAVAIR  4.10  makes  it  difficult  for  programs  to  step  outside  of  the  parameters  of 
past  source  selections.  There  are  positive  and  negative  aspects  of  this  set  of  circumstances.  On 
the  positive  side,  source  selection  is  a  controlled  process,  structured  in  a  manner  that  avoids 
protests  from  offerors  and  results  in  a  fair  and  equitable  evaluation  of  proposals.  On  the  negative 
side,  innovation  is  dampened,  with  incremental  improvement  being  the  second  most  likely 
outcome  of  change  efforts.  The  most  likely  outcome  is  maintenance  of  the  status  quo.  Step- 
gains  in  improvement  through  utilization  of  truly  unique  or  different  approaches  to  source 
selection  are  unlikely. 

The  next  Chapter  will  further  develop  the  analysis  of  points  noted  in  the  case  studies 
outlined  in  Chapter  III.  The  outcome  of  this  analysis  will  be  applied  to  Chapter  IV  observations 
concerning  source  selection  at  NAVAIR.  Conclusions  will  then  be  drawn  concerning 
implementation  of  the  proposed  multi-phase  source  selection  strategy  at  NAVAIR. 
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V. 


SUMMARY,  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  SUMMARY  OF  PREVIOUS  CHAPTERS 

As  a  first  step  in  organizing  points  noted  by  each  program,  groupings  can  made  with  the 
following  categories;  expected  benefits,  barriers  to  use,  challenges  and  advantages.  Prior  to 
analyzing  the  points  noted  for  pertinence  to  either  source  selection  or  NAVAIR,  they  will  be 
grouped’ to' identify  common  themes. 

1.  Expected  Benefits  From  Use  of  Intended  Strategy 

Figure  3.1  categorizes  the  expected  benefits  from  the  use  of  intended  strategy  in  each 
program.  The  benefits  were  explained  fully  in  the  individual  program  write-ups.  They  are 
represented  in  Figure  3.1  by  brief  descriptive  bullets.  The  munbered  rows  represent  cases  where 
similar  points  were  made  in  more  than  one  program.  In  the  case  of  expected  benefits,  there  are 
four  such  rows.  The  unnumbered  rows  depict  expected  benefits  in  individual  programs  that  do 
not  necessarily  correspond  to  others  in  the  same  row. 

The  first  grouping  of  expected  benefits  points  out  the  shared  expected  benefit  of  the 
integration  of  commercial  technologies  into  Government  programs.  The  second  grouping  of 
expected  benefits  points  out  the  shared  expected  benefit  of  improved  communication  between 
industry  and  Government.  The  third  grouping  of  expected  benefits  points  out  the  shared 
expected  benefit  of  cost  reduction.  The  fourth  grouping  of  expected  benefits  points  out  the 
shared  expected  benefit  of  increased  iimovation  in  requirement  refinement,  solution  options  and 
system  design. 

Other  expected  benefits  that  did  not  fall  into  natural  groupings  are  similar  to  the  goals  of 
the  proposed  multi-phase  source  selection  strategy.  These  include  the  empowered  industry 
expected  in  the  Deepwater  program.  The  MCASMP  had  as  expected  benefits  flexibility  in 
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defining  the  scope  of  work  and  risk  mitigation  through  the  incremental  commitment  of  funds. 
An  expected  benefit  of  the  Arsenal  Ship  program  was  a  shortened  source  selection  schedule. 
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Figure  3.1.  Summary  of  Expected  Benefits  From  Use  of  Intended  Strategies 

(prepared  by  author) 
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2. 


Barriers  to  Use  of  Intended  Strategies 


Figure  3.2  categorizes  the  barriers  to  use  of  the  intended  strategy  in  each  program.  The 
barriers  were  explained  fully  in  the  individual  program  write-ups.  They  are  represented  in 
Figure  3.2  by  brief  descriptive  bullets.  The  numbered  rows  represent  cases  where  similar  points 
were  made  in  more  than  one  program.  In  the  case  of  barriers,  there  are  five  such  rows.  The 
xmnumbered  rows  depict  barriers  in  individual  programs  that  do  not  necessarily  correspond  to 
others  in  the  same  row. 

The  first  grouping  of  barriers  points  out  the  shared  barrier  of  the  traditional  acquisition 
approach  and  developed  mindset  of  participants  in  the  process.  The  second  grouping  of  barriers 
points  out  the  shared  barrier  of  organizational  resistance  to  change.  The  third  grouping  of 
barriers  points  out  the  shared  barrier  of  difficulties  associated  with  funding.  The  fourth  grouping 
of  barriers  points  out  the  shared  barrier  of  the  length  of  phases.  This  barrier  is  usually  driven  by 
external  events,  such  as  funding.  This  barrier  can  result  in  concxirrency  when  concurrency  is  not 
appropriate.  The  fifth  grouping  of  barriers  points  out  the  shared  barrier  of  regulatory  guidance. 

Other  barriers  that  did  not  fall  into  natural  groupings  are  similar  to  barriers  expected 
during  execution  of  the  proposed  multi-phase  source  selection  strategy.  These  barriers  include 
The  Deepwater  barrier  of  limiting  competition  in  follow-on  phases  and  the  MCASMP  barrier  of 
requirement  and  solution  refinement. 
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Figure  3.2.  Summary  of  Barriers  To  Use  of  Intended  Strategies  (prepared  by  author) 


3.  Challenges  of  Strategies  Used 


Figure  3.3  categorizes  the  challenges  of  strategies  used  present  in  each  program.  The 
challenges  were  explained  fully  in  the  individual  program  write-ups.  They  are  represented  in 
Figure  3.3  by  brief  descriptive  bullets.  The  numbered  rows  represent  cases  where  similar  points 
were  made  in  more  than  one  program.  In  the  case  of  challenges,  there  are  seven  such  rows.  The 
unnumbered  rows  depict  challenges  in  individual  programs  that  do  not  necessarily  correspond  to 
others  in'the  same  row.  *" 

The  first  grouping  of  challenges  points  out  the  shared  challenge  of  obtaining  contractor 
investment  in  the  early  phases  of  a  program.  The  second  grouping  of  challenges  points  out  the 
shared  challenge  of  accounting  and  allocating  funding  and  costs.  The  third  grouping  of 
challenges  points  out  the  shared  challenge  of  program  office  staffing.  The  fourth  grouping  of 
challenges  points  out  the  shared  challenge  of  working  with  a  systems  integrator.  The  fifth 
grouping  of  challenges  points  out  the  shared  challenge  of  working  with  contractors  throu^ 
repeated  down  selects.  The  sixth  grouping  of  challenges  points  out  the  shared  challenge  of 
staffing  program  offices  with  personnel  with  appropriate  skill  mixes  including  a  propensity  for 
accepting  new  methods  of  doing  business.  The  seventh  grouping  of  challenges  points  out  the 
shared  challenge  of  working  outside  of  strict  regulatory  guidance. 

Other  expected  challenges  that  did  not  fall  into  naftnal  groupings  are  similar  to 
challenges  expected  in  the  proposed  multi-phase  source  selection  strategy.  These  challenges 
include  the  MCASMP  challenge  of  using  PPI  as  a  true  discriminator,  the  JDAM  requirement  of 
the  Government  thinking  carefully  through  source  selection  and  building  control  mechanisms  to 
shape  Government  and  contractor  behavior,  and  three  of  the  Arsenal  Ship  program  challenges. 
The  three  Arsenal  Ship  challenges  are  poor  understanding  of  the  magnitude  of  development  tasks 
required,  constrained  trade  space  and  the  stigma  of  fixed  price  development. 
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Figure  3.3.  Summary  of  Challenges  of  Strategies  Used  (prepared  by  author) 
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4. 


Advantages  of  Strategies  Used 


Figure  3.4  categorizes  the  advantages  of  strategies  used  in  each  program.  The  advantages 
■were  explained  fully  in  the  individual  program  write-ups.  They  are  represented  in  Figure  3.4  by 
brief  descriptive  bullets.  The  numbered  rows  represent  cases  where  similar  points  were  made  in 
more  than  one  program.  In  the  case  of  advantages,  there  are  five  such  rows.  The  unnumbered 
rows  depict  advantages  in  individual  programs  that  do  not  necessarily  correspond  to  others  in  the 
same  row. 

The  first  grouping  of  advantages  points  out  the  shared  advantage  of  enabling  the  use  of  a 
smaller  program  office  exercising  streamlined  oversight.  The  second  grouping  of  advantages 
points  out  the  shared  advantage  of  minimizing  the  potential  for  protests.  The  third  grouping  of 
advantages  points  out  the  shared  advantage  of  integration  of  commercial  technology.  The  fourth 
grouping  of  advantages  points  out  the  shared  advantage  of  cost  control.  The  fifth  grouping  of 
advantages  points  out  the  shared  advantage  of  improved  contractor  focus  and  effort. 

Other  advantages  that  did  not  fall  into  natural  groupings  are  similar  to  advantages  sought 
through  use  of  the  proposed  multi-phase  source  selection  strategy.  These  advantages  include  the 
Deepwater  advantages  of  empowered  industry  and  cost  sharing  in  system  development.  Also 
included  are  the  MCASMP  advantages  of  flexibility  in  defining  the  scope  of  work  and  risk 
mitigation  through  the  incremental  commitment  of  funds.  The  Arsenal  Ship  program  advantages 
of  leveraging  commercial  practices  and  technologies  and  a  shortened  source  selection  schedule 
are  also  included.  The  only  reason  that  the  Deepwater  program  was  not  cited  as  having  achieved 
these  last  two  advantages  of  the  Arsenal  Ship  program  is  that  the  program  has  not  progressed  far 
enough  yet.  It  is  fully  anticipated  that  the  Deepwater  program  will  have  these  same  advantages. 
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Figure  3.4.  Summary  of  Advantages  of  Strategies  Used  (prepared  by  author) 
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B.  POINTS  NOTED  IN  CASE  STUDIES  THAT  MATCH  ELEMENTS  OF  MULTI¬ 

PHASE  SOURCE  SELECTION 


1.  Deepwater 

Deepwater  expected  a  benefit  of  empowered  industry  with  the  flexibility  to  leverage 
proven  technologies  and  new  processes  to  maximize  effectiveness.  This  is  an  expected  benefit 
of  the  multi-phase  source  selection  approach.  Initially  evaluating  offerors  on  PPI  and  a 
capability  statement  while  allowing  for  requirement  refinement  allows  maximum  leveraging  and 
incorporation  of  innovative  and  commercial  solutions. 

Deepwater  expected  to  reduce  risk  through  collaborative  teaming  between  industry  and 
Government.  A  main  thrust  of  the  multi-phase  source  selection  is  to  increase  and  enhance 
communication  between  industry  and  Government.  This  enhanced  communication  is  expected 
to  reduce  risk. 

In  Deepwater,  the  phase  I  source  selection  for  conceptual  design  resulted  in  the  award  of 
fixed-price  contracts  to  offerors.  The  proposed  multi-phased  source  selection  aims  to  achieve 
this  same  result  through  the  improvement  of  up-ffont  planning  and  risk  reduction. 

Working  with  industry  teams  through  a  systems  integrator  lengthened  communication 
lines,  a  challenge  to  the  Deepwater  source  selection  strategy.  These  lengthened  communication 
lines  hindered  the  free  flow  and  active  interchange  of  ideas  and  potential  solutions.  A  goal  of  the 
proposed  multi-phase  source  selection  is  to  have  open  communications  with  industry  before 
award  and  through  execution  of  the  contract. 

2.  Arsenal  Ship 

The  Arsenal  Ship  program’s  intended  methodology  of  planning  smaller  pieces  by  way  of 
phases  matches  the  proposed  multi-phase  source  selection  strategy.  In  the  case  of  Arsenal  Ship, 


85 


not  allocating  enough  time  for  planning  for  follow-on  phases  throughout  the  phases  hindered  the 
success  possible  through  the  use  of  smaller  increments.  The  multi-phase  source  selection 
emphasizes  increased  planning  time  and  effort  to  achieve  full  understanding  of  the  requirement 
by  all  parties.  This  increased  understanding  is  intended  as  an  enabler  to  better  understand  the 
timelines  needed  and  overcome  this  difficulty.  Timeline  constraints  driven  by  external  forces 
such  as  funding  or  budgeting  has  the  possibility  to  cause  this  same  difficulty,  in  spite  of  fuller 
understanding  of  developmental  time  requirements. 

The  use  of  OTA  in  the  Arsenal  Ship  program  was  intended  to  provide  the  opportunity  to 
create  unique  system  concepts  through  design  tradeoffs  within  a  larger  solution  space  than  a 
traditional  approach  allows.  Relaxation  of  all  acquisition  regulations  is  not  a  part  of  the  multi¬ 
phase  source  selection,  however,  it  is  recognized  that  select  regulatory  guidance  will  be  required 
to  be  changed  to  fully  implement  the  proposed  multi-phase  source  selection. 

One  difference  of  the  proposed  multi-phase  source  selection  strategy  is  using  a  capability 
statement  vice  a  full-blown  proposal  to  evaluate  offerors.  The  Arsenal  Ship  program  evaluated 
phase  I  offerors  in  part  on  how  well  the  offeror  demonstrated  its  capability.  The  Arsenal  Ship 
program  allowed  industry  to  set  requirements,  which  increased  design  innovation.  Evaluating 
offerors  on  the  basis  of  a  capability  statement  in  the  multi-phase  source  selection  strategy 
facilitates  this  same  outcome. 

The  underfunding  of  the  Arsenal  Ship  program  presented  a  significant  barrier  to  success. 
This  underfunding  was  due  to  two  factors;  inadequate  analysis  of  required  developmental 
resources  prior  to  program  initiation  and  poor  understanding  of  the  magnitude  of  developmental 
tasks  required.  The  proposed  multi-phase  source  selection  process  is  intended  to  fix  this  with 
increased  focus  on  more  intense  up  front  planning  and  collaborative  Government  and  industry 
effort. 


The  abbreviated  timeline  of  the  Arsenal  Ship  program  drove  parallel  completion  of  both 
design  and  pricing.  As  a  result,  estimates  were  inconsistent  and  lacked  depth.  In  the  proposed 
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multi-phase  source  selection  strategy,  the  difficulty  of  not  really  understanding  either  the 
program  risk  or  what  is  needed  from  a  technology  standpoint  is  addressed.  Planning  and  talking 
with  industry  is  key  to  resolution  of  this  difficulty. 

An  expected  benefit  in  the  Arsenal  Ship  program  was  that  continuous  interaction  with 
industry  teams  throughout  the  design  phases  would  shorten  the  source  selection  schedule.  This 
matches  the  proposed  multi-phase  source  selection’s  intended  outcome. 

3.  Marine  Corps  Aviation  Systems  Master  Plan 

In  the  MCASMP,  one  intention  of  the  source  selection  strategy  employed  was  to 
mutually  evolve  the  user’s  requirements  in  a  dedicated  analysis  and  definition  phase. 

Participants  in  this  phase  included  the  users,  industry  and  the  program  office.  In  the  proposed 
multi-phase  source  selection  strategy  one  goal  is  to  evolve  the  user’s  requirements,  preferably 
before  contract  award  or  during  phase  I,  before  committing  to  larger  dollar  contracts. 

The  MCASMP  program  office  saw  system  analysis  and  the  design  phase  as  a  means  to 
expand  design  space  in  their  program.  System  cost  and  performance  tradeoffs  needed  to  be 
accommodated  early  on.  This  matches  the  goals  and  strategy  of  the  proposed  multi-phase  source 
selection. 

In  the  MCASMP,  the  nature  of  the  requirement  required  evolution  of  the  requirement 
during  its  definition.  This  is  also  seen  as  a  necessity  in  the  proposed  multi-phase  source 
selection  strategy.  The  ongoing  requirement  and  solution  refinement  during  the  execution  of 
contracts  in  the  MCASMP  program  hampered  the  Government’s  ability  to  compete  contracts. 
This  is  another  reason  that  supports  using  the  proposed  multi-phase  source  selection  strategy. 

An  expected  benefit  of  the  strategy  employed  by  the  MCASMP  program  was  using  the 
incremental  commitment  of  fimds  to  mitigate  risk.  This  is  similar  to  the  strategy  employed  in  the 
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proposed  multi-phase  source  selection,  where  phases  are  funded  and  treated  as  separable  parts  of 
the  process. 

In  the  MCASMP  program,  evaluation  of  PPI  was  problematic.  Evaluating  offerors  on 
past  performance  is  key  to  utilizing  the  proposed  multi-phase  source  selection  strategy. 

4.  Joint  Direct  Attack  Munition 

The  JDAM  Program  used  long-term  prime/subcontractor  relationships  as  a  key  business 
practice  to  drive  behaviors.  This  heightened  and  reinforced  the  use  of  past  performance  as  an 
evaluation  tool.  The  proposed  multi-phase  source  selection  strategy  also  proposes  long-term 
relationships  with  industry  and  the  use  of  past  performance  as  key  component  of  offeror 
evaluation. 

A  barrier  noted  by  the  JDAM  program  manager  was  the  inability  to  put  together  an 
experienced,  innovative  team.  This  same  inability,  if  present,  could  hamper  the  ability  to  use  and 
maximize  the  potential  benefits  of  the  proposed  multi-phase  source  selection  strategy. 

In  the  JDAM  program,  the  notion  of  a  rolling  downselect  was  envisioned  as  a  means  to 
enhance  the  level  and  focus  of  communication  between  the  Government  and  industry  during 
evaluation  and  performance  periods.  The  proposed  multi-phase  source  selection  strategy  serves 
as  a  means  by  which  to  achieve  this  same  heightened  level  of  clarity  in  communication  with 
industry. 

Implementing  the  rolling  downselect  process  in  the  JDAM  program  required  the 
Government  to  carefully  think  through  what  was  most  important  in  making  source  selections  and 
to  perform  evaluations  using  the  down  select  criteria  prior  to  the  actual  down  select.  This  is  a 
key  element  in  evaluating  past  performance,  which  is  essential  to  success  with  the  proposed 
multi-phase  source  selection  strategy. 
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The  JDAM  program  implemented  CATV,  which  assumed  costs  could  be  controlled  if 
affordability  was  introduced  early  in  the  development  process  as  a  key  requirement.  The 
proposed  multi-phase  source  selection  strategy  includes  this  type  of  tradeoff  in  its  process. 

Before  the  start  of  phase  I  HMD  in  the  JDAM  program,  the  program  office  conducted  a 
survey  of  conunercial  companies  to  see  what  practices  prevailed  in  their  relationships  with  key 
suppliers.  A  key  element  of  the  proposed  multi-phase  source  selection  strategy  is  to  understand 
industry  before  issuing  a  contract. 

The  JDAM  program  office  allowed  contractors  control  over  technical  requirements  and 
the  TDP.  Contractors  modified  the  TDP  as  needed  to  control  costs,  and  were  not  required  to 
disclose  trade  secrets,  as  long  as  they  met  key  performance  requirements.  The  proposed  multi¬ 
phase  source  selection  strategy  emphasizes  a  similar  strategy.  The  use  of  dedicated  Government 
teams  for  each  industry  partner,  one  method  espoused  in  the  multi-phase  source  selection,  also 
serves  to  ensure  that  innovative  ideas  developed  by  one  source  are  not  inadvertently  transferred 
to  competing  sources. 

In  the  JDAM  program,  performance  in  a  prior  phase  played  a  critical  role  in  the  PPI 
evaluation  when  selecting  contractors  for  follow-on  phases.  One  element  of  the  proposed  multi¬ 
phase  source  selection  strategy  is  increased  communications  driven  in  part  by  concern  over  using 
past  performance  in  an  earlier  phase  to  select  for  the  next  phase. 

Open  and  clear  communications  and  down  select  criteria  enabled  JDAM  EMD  contracts 
to  be  awarded  with  30  percent  less  Government  effort  and  with  bid  and  proposal  costs  reduced 
by  50  percent.  Similar  outcomes  are  expected  through  utilization  of  the  proposed  multi-phase 
source  selection  strategy. 

Success  with  the  proposed  multi-phase  source  selection  strategy  requires  changes  in 
incentives  used  and  the  way  acquisition  practitioners  think  about  them.  The  structured  feedback 
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of  the  rolling  down  select  process  utilized  in  the  JDAM  program  proved  to  motivate  and  focus 
the  contractor  on  the  right  issues  better  than  incentives  or  award  fees. 

One  goal  of  PBA  efforts  is  to  bring  in  commercial  success  stories.  JDAM  components 
are  approximately  85  percent  commercial.  Using  COTS  greatly  reduced  the  time  and  expense  of 
developing  JDAM.  The  proposed  multi-phase  source  selection  strategy  has  the  ability  to 
facilitate  the  use  of  commercial  items  in  the  same  manner  that  the  JDAM  program  did.  One  of 
the  main  ways  that  the  JDAM  program  achieved  this  high  level  of  commercial  access  was  by 
minimizing  the  use  of  military  regulations  and  specifications. 

C.  POINTS  NOTED  IN  CASE  STUDIES  CONTRASTED  WITH  NAVAIR  SOURCE 

SELECTION 

The  four  groupings  of  advantages  noted  in  Figure  3.4  are  desired  in  NAVAIR  source 
selections.  As  noted  in  Chapter  IV,  a  significant  driving  factor  in  NAVAIR  source  selection  is 
reducing  the  risk  of  protest.  This  driving  factor  receives  more  attention  from  decision-makers  in 
source  selection  strategy  formulation  than  other  elements.  For  this  reason,  this  discussion  places 
the  most  emphasis  on  this  particular  advantage. 

The  advantages  of  protest  avoidance  noted  by  both  the  JDAM  and  Arsenal  Ship  programs 
mesh  with  the  strongly  emphasized  goal  of  protest  avoidance  in  NAVAIR  source  selection.  In 
NAVAIR  source  selections,  the  methodology  chosen  to  achieve  an  acceptable  level  of  risk  in 
terms  of  protest  avoidance  has  been  to  rigidly  standardize  and  uniformly  apply  a  formal  source 
selection  process.  The  proposed  multi-phase  source  selection  facilitates  protest  avoidance  in  a 
similar  manner  to  that  demonstrated  in  the  Arsenal  Ship  and  JDAM  programs.  The  Arsenal 
Ship,  JDAM  and  multi-phase  approaches  share  the  element  of  increased  openness  in  the  award 
process.  Increased  feedback  to  contractors  and  offerors  during  selection  for  participation  in  and 
execution  of  phases  serves  to  dissuade  protests. 
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Of  significance  to  using  the  proposed  multi-phase  source  selection  strategy  at  NAVAIR 
is  the  identification  of  barriers  to  use.  Barriers  must  first  be  identified  before  they  can  be 
addressed. 

Referring  to  Figure  3.2,  the  first  two  groupings  of  barriers  are  significant  factors  at 
NAVAIR.  The  usual  method  of  doing  business  with  regard  to  source  selection  is  clearly  laid  out 
and  strictly  adhered  to.  This  usual  method  of  doing  business  is  entrenched  in  the  mindset  of 
decision-makers  as  being  the  right  answer  for  if  not  all,  then  certainly  a  predominant  majority  of 
source  selections  for  major  systems.  The  resulting  organizational  inertia  to  considering  the  use 
of  alternate  strategies  was  noted  in  the  MCASMP  as  a  barrier  to  tailoring  the  source  selection 
strategy  employed.  When  considering  the  use,  even  on  a  pilot  project  basis,  of  the  proposed 
multi-phase  source  selection,  this  organizational  resistance  to  alternative  strategies  may  inhibit 
appropriate  selection  and  application  of  source  selection  strategy. 

In  three  of  the  fom  cases  presented  in  this  thesis,  regulatory  guidelines  were  noted  as  a 
barrier  to  innovation  in  source  selection.  In  considering  opportunities  to  use  the  proposed  multi¬ 
phase  source  selection  strategy  at  NAVAIR,  this  same  barrier  is  also  present.  The  NAVAIR 
source  selection  instruction  (Ref  28)  provides  guidance  for  tailoring  and  utilizing  other  than  the 
formal  source  selection  methodology,  but  not  for  major  systems  acquisition.  The  multi-phased 
source  selection  strategy  as  proposed  also  requires  exemption  from  higher-level  statutory  and 
regulatory  guidance.  Because  of  this,  the  proposed  multi-phase  source  selection  strategy  can 
only  be  used  with  specific  authority  to  deviate  from  published  statutory  and  regulatory  guidance. 
In  the  case  of  Deepwater  and  JDAM,  the  approach  used  was  to  obtain  specific  authority  to 
deviate  from  regulatory  guidance.  In  the  case  of  the  Arsenal  Ship  program,  blanket  authority 
was  obtained  to  deviate  from  regulatory  guidance  through  the  use  of  OTA.  In  the  case  of  the 
MCASMP,  authority  to  deviate  from  regulatory  guidance  was  not  sought  and  as  a  result,  source 
selection  strategy  options  were  limited. 

In  both  the  Arsenal  Ship  and  Deepwater  programs,  the  length  of  phases  was  noted  as  a 
barrier  to  the  use  of  the  chosen  source  selection  strategy.  External  factors  such  as  timelines 
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driven  by  funding  cycles  were  a  key  driver  in  shortening  phases.  Addressing  the  stability  and 
commitment  of  funds  to  a  program  is  beyond  the  scope  of  this  thesis.  A  second  significant 
factor,  arguably  the  most  significant  in  using  the  proposed  multi-phase  source  selection  strategy, 
is  the  ability  to  dedicate  the  needed  time  for  early  phases.  In  the  early  phases,  requirements  are 
being  refined,  solutions  are  being  developed,  and  significant  unknowns  are  present.  As  seen  in 
the  Arsenal  Ship  program,  tight  timelines  in  early  phases  can  result  in  contractors  preparing 
proposals  for  follow-on  phases  concurrent  with  completing  developmental  work  and  before 
reaching  a  full  understanding  of  all  elements  needed  to  make  accurate  estimates  and  predictions 
about  future  capabilities,  costs  and  schedules.  In  the  Deepwater  program,  these  same  elements 
resulted  in  an  extension  of  the  first  phase  in  time,  scope  and  dollar  value.  A  key  tenet  of  the 
proposed  multi-phase  source  selection  strategy  is  investing  more  time  early  on  in  the  program  to 
fully  understand  the  requirements  and  proposed  solutions.  The  implication  for  source  selection 
at  NAVAIR  is  not  to  utilize  a  source  selection  strategy  whose  success  depends  on  a  significant 
up-ffont  investment  of  time  when  it  is  known  that  external  factors  will  force  the  abbreviation  or 
compression  of  schedule. 

In  contrast  to  the  need  for  adequate  time  early  in  the  source  selection  phases  is  the 
advantage  noted  in  the  Arsenal  Ship  program  of  a  shortened  source  selection  schedule. 
Continuous  interaction  with  industry  teams  throughout  the  design  phases  shortened  the  source 
selection  schedule  when  compared  to  a  traditional  acquisition  approach.  In  the  proposed  multi- 
phased  source  selection,  this  heightened  level  of  interaction  is  also  present.  Offsetting  the  need 
to  invest  adequate  time  for  early  phases,  the  multi-phase  strategy  has  the  capability  to  shorten  the 
evaluation  and  selection  of  contractors  when  compared  to  a  more  traditional  approach.  The 
recent  revision  of  regulatory  guidance  as  embodied  in  DoD  Directive  5000.1  complements  this 
approach  in  emphasizing  not  starting  acquisition  programs  until  requirements  are  clearly  defined 
and  understood,  followed  by  an  incremental  or  evolutionary  acquisition  strategy.  The  proposed 
multi-phase  source  selection  strategy  is  a  good  fit  inside  of  this  framework. 
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D.  CONCLUSIONS 


1.  Multi-Phase  Source  Selection  Advantages 

Using  multiple  phases  in  source  selection  has  distinct  advantages.  These  advantages 
include  freer  communication  between  industry  and  Government,  especially  during  the  pre¬ 
proposal  stage.  Increased  emphasis  is  placed  on  up  front  planning.  Increased  discussions  as 
early  as  the  requirements  phase  are  facilitated.  Government- and  contractors  have  a  clear  and  ' ' 
unambiguous  understanding  of  the  requirement,  industry’s  ability  to  meet  that  requirement,  and  a 
company’s  specific  approach  for  doing  so.  Industry  can  help  the  Government  tailor  requirements 
to  achieve  realistic  outcomes  vice  failing  to  achieve  overoptimistic  outcomes  promised  in  the 
process  of  commimicating  through  constrained  channels.  This  increased  understanding  has  the 
potential  to  reduce  the  level  of  risk  to  where  fixed-price  contracts  maybe  an  acceptable  risk- 
allocation  method.  Regardless  of  the  potential  for  use  of  cost  or  PBA,  the  increased  level  of 
understanding  achieved  facilitates  risk  being  more  appropriately  allocated  between  the 
Government  and  its  suppliers. 

The  Government  may  continue  to  evolve  its  requirements  during  source  selection.  If  the 
demonstration/validation  phase  is  structured  so  that  prototypes  of  competing  systems  are 
produced  and  tested,  it  is  possible  to  make  design  selection  from  the  competing  separate  designs 
without  sharing  information.  Changes  to  Government  requirements  that  occur  during  refinement 
of  requirements  do  not  have  to  be  re-competed.  This  protects  intellectual  information,  as  well  as 
promoting  industry  innovation. 

By  aligning  business  practices  more  closely  with  those  in  use  in  the  commercial  sector, 
the  Government  gains  increased  access  to  available  technology. 
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2. 


Multi-Phase  Source  Selection  Risks 


Risks  must  be  addressed  in  considering  the  use  of  the  proposed  multi-phase  source 
selection  strategy.  These  risks  include  difficulties  inherent  in  evaluating  sources  on  the  basis  of 
past  performance  and  capabilities.  In  the  case  of  MCASMP,  PPI  was  ambiguous  enough  to 
make  use  of  PPI  as  a  true  discriminator  difficult  in  source  selection.  While  progress  has  been 
made  in  refinement  of  the  use  of  PPI  in  source  selection,  difficulties  still  exist.  Evaluating 
offerors  on  the  basis  of  a  capability  statement  raises  concern  about  ability  to  perform.  Firms  well 
versed  in  design  work  and  proposal  preparation  could  possibly  prevail  through  multiple  phases 
of  source  selection.  Early  phase  offeror  evaluation  may  focus  more  heavily  on  past  performance 
in  executing  instant  contracts  that  deal  with  design.  Offerors  could  make  the  case  that  PPI  on 
production  contracts  is  not  pertinent  at  this  point.  If  the  design  oriented  firms  prevail,  follow  on 
phases  leading  to  production  could  be  limited  to  competition  among  firms  less  suited  to 
production. 

Risk  is  present  in  the  form  of  potential  for  inadvertently  transferring  innovative  ideas 
developed  by  one  source  to  competitors.  Restriction  and  regulation  of  communication  between 
industry  and  Government  in  traditional  acquisitions  is  a  result  of  efforts  to  reduce  this  risk. 
Increased  communications  between  industry  and  Government  with  fewer  restrictions  is  a 
hallmark  of  the  multi-phase  source  selection.  Flexibility  in  communication  and  a  shift  in 
thinking  about  faimess  when  dealing  with  offerors  is  required  to  mitigate  this  risk. 

Maintaining  design  competition  is  costly.  As  seen  in  the  MCASMP  program,  carrying 
competition  incurs  costs  that  the  program  may  not  be  able  or  willing  to  sustain.  This  presents  the 
tradeoff  risk  of  the  benefits  versus  the  cost  of  competition.  Achieving  better  pricing  for  similar 
performance  is  facilitated  in  the  multi-phase  source  selection  process  through  competition.  The 
cost  of  this  competition  needs  to  be  weighed  against  the  benefits  of  improved  performance  or 
reduced  cost. 
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PBA  is  a  controversial  subject.  The  DoD  has  encountered  serious  difficulties  using 
fixed-price  development  contracts  in  the  past.  The  corporate  memory  of  those  difficulties  serves 
to  stigmatize  use  or  proposed  use  of  fixed-price  development  contracts. 

3.  Multi-Phase  Source  Selection  and  Price-Based  Acquisition 

The  proposed  multi-phase  source  selection  strategy  facilitates  the  use  of  PBA  in  several 
ways:  Use  of  the  multi-phase  source  selection  aids  in  the  development  of  an  appropriate 
business  case  analysis  for  specific  acquisitions.  Enhanced  communication  results  in  more 
thorough  understanding  of  requirements,  solutions  and  risks.  Multi-phase  source  selection  thus 
alters  the  risk  equation,  reducing  risks  associated  with  source  selection  for  major  systems. 

Multi-phase  source  selection  aligns  acquisition  practices  more  closely  with  those  in  use  in 
the  commercial  sector,  for  example,  by  not  requiring  changes  to  requirements  to  be  re-competed. 
Firms  that  were  previously  not  interested  in  doing  business  with  the  Government  may  be  more 
inclined  to  do  so  as  the  result  of  this  change.  This  broadens  opportunities  to  bring  existing 
commercially  available  technology  to  bear  against  risks. 

4.  Multi-Phase  Source  Selection  at  the  Naval  Air  Systems  Command 

Actions  taken  to  mitigate  risk  tend  to  drive  the  entire  acquisition  and  set  the  stage  for 
predominant  use  of  a  formal  source  selection  process  in  major  systems  acquisitions  at  NAVAIR. 

Utilization  of  the  proposed  multi-phase  source  selection  strategy  at  NAVAIR  should  be 
done  in  acquisitions  where  there  is  a  high  degree  of  confidence  in  the  ability  to  determine  a  fair 
and  reasonable  price  without  obtaining  supplier  cost  data.  Multi-phase  source  selection  also  fits 
when  the  requirement  is  fluid  and  is  likely  to  change  significantly  after  information  has  been 
exchanged  with  potential  sources.  A  good  indicator  of  fit  is  the  presence  of  two  or  more  sources 
expected  to  be  capable  of  meeting  the  requirements.  The  presence  of  competitors  is  not  an 
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absolute  requirement.  Competition  enables  PBA,  as  opposed  to  being  a  prerequisite,  as  seen  in 
the  discussion  in  Chapter  II. 

If  resources  are  available,  contractor-specific  Government  teams  can  be  formed.  This 
approach  was  utilized  with  great  success  in  the  JDAM  program.  This  type  of  team  assignment  is 
not  absolutely  necessary,  and  may  not  be  appropriate  when  there  is  insufficient  expertise  or 
resources  to  support  the  effort.  As  seen  in  Chapter  IV,  NAVAIR  has  experienced  difficulty  in 
consistently  developing  and  assigning  personnel  experienced  with' offeror  PPI  evaluation. 

Similar  difficulties  may  be  encountered  in  forming  dedicated  teams  for  multiple  offerors. 

Depending  upon  the  length  of  time  and  effort  expected  to  take  to  complete  the  early 
phases,  the  Government  may  choose  to  fund  them  in  part  or  in  whole.  If  funding  is  tight,  the 
Government  may  take  extra  effort  to  obtain  contractor  investment  in  early  phases.  The  use  of 
multiple  phases  in  source  selection  can  facilitate  this,  as  it  did  in  the  Deepwater  program. 

E.  RESEARCH  QUESTIONS  AND  ANSWERS 

1.  Secondary  Research  Question  #1:  What  is  a  multi-phase  source  selection 

strategy,  and  how  does  it  differ  fi'om  present  source  selection  strategies? 

The  proposed  multi-phase  source  selection  strategy  initially  selects  sources  determined  to 
be  the  most  qualified  by  evaluating  offeror  capability  statements.  The  capability  statement 
includes  limited  information  on  past  performance,  ability  to  perform,  conceptual  approach,  and  a 
rough  order  of  magnitude  price.  Government  requirements  are  initially  stated  as  objectives.  The 
Government  works  with  selected  sources  to  refine  requirements.  Major  consideration  is  given  to 
reducing  program  risk  and  taking  maximum  advantage  of  commercially  available  technology. 
Requirement  refinement  may  be  done  through  one  or  more  phases.  The  Government  may  fund 
all  or  a  part  of  this  phase  of  the  development.  Requirement  refinement  continues  until  they  are 
understood  well  enough  by  both  Government  and  industry  to  allow  sources  to  propose  a  firm- 
fixed  price  for  performance.  In  each  successive  cycle,  the  Government  may  elect  to  reduce  the 
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number  of  sources.  In  the  proposal  development  and  evaluation  phase,  the  only  sources  eligible 
to  participate  are  those  selected  on  the  basis  of  their  initial  submission.  If  a  requirements 
development  phase  is  conducted,  sources  may  be  limited  to  those  participating  in  the  proposal 
development  and  evaluation  phase.  No  J&A  is  required  for  limiting  competition  in  this  manner. 
The  solicitation  provides  notice  to  offerors  of  the  extent  to  which  the  Government  reserves  the 
right  to  negotiate  requirements  or  terms  or  conditions  different  from  those  stated  in  the 
solicitation  without  amending  it.  Award  can  be  made  to  single  or  multiple  offerors.  Following 
receipt  of  proposals,  one  or  more  offerors  may  be  eliminated  during^  the' evaluation.  •  ' 

This  strategy  differs  from  present  source  selection  strategies  in  two  ways.  First,  changes 
to  requirements  do  not  have  to  be  re-competed.  Changes  to  the  requirement  may  occur  as  it  is 
being  refined.  This  may  happen  during  a  contract  or  not  during  a  contract.  Second,  no  J&A  is 
required  for  other  than  full  and  open  competition  when  going  into  a  follow-on  phase  considering 
only  offerors  who  were  included  in  prior  phases. 

2.  Secondary  Research  Question  #2:  How  does  the  multi-phase  source  selection 
strategy  recommended  in  the  PBA  report  dated  December  1999  differ  from  multi-phase 
strategies  currently  utilized  at  NAVAIR? 

The  case  study  presented  in  Chapter  HI  on  the  MCASMP  is  an  example  of  a  NAVAIR 
source  selection  strategy.  This  case  was  included  more  as  an  example  of  differences  between  as 
executed  and  as  desired  than  as  a  stand-alone  example  of  a  multi-phased  source  selection.  The 
JDAM  program  represents  a  joint  program  office  between  the  Air  Force  and  the  Navy.  The 
Navy  component  was  supplied  by  NAVAIR.  Therefore,  it  can  be  said  that  NAVAIR  utilizes 
multi-phase  source  selection  strategies  as  presented  in  the  case  studies  in  Chapter  III. 

As  noted  in  Chapter  FV,  however,  the  prevailing  organizational  culture  at  NAVAIR 
favors  a  traditional  source  selection  strategy  in  major  systems  acquisition.  In  this  sense,  the 
multi-phase  source  selection  represents  a  fairly  radical  shift  in  approach  from  the  predominant 
model  in  use. 
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3.  Secondary  Research  Question  #3:  To  what  extent  does  use  of  a  multi-phase 
source  selection  strategy  facilitate  PBA? 


The  most  significant  way  in  which  the  multi-phase  source  selection  strategy  facilitates 
PBA  is  through  increased  communication  between  Government  and  industry,  specifically  in  the 
pre-proposal  stage.  This  increased  communication,  intended  to  facilitate  better  understanding  of 
requirements  by  both  Government  and  industry,  reduces  risk  to  the  point  that  offerors  can  submit 
firm-fixed  price  proposals. 

4.  Secondary  Research  Question  #4:  What  are  the  potential  benefits  and  risks  of 

utilizing  a  multi-phase  source  selection  strategy? 

Potential  benefits  of  using  a  multi-phase  source  selection  strategy  include  freer 
communications  between  industry  and  Government.  This  facilitates  several  outcomes:  a 
reduction  in  risk  through  better  understanding  of  requirements,  industry  tailoring  requirements  to 
meet  Government  needs  and  the  use  of  fixed-price  contracts.  Other  benefits  include  an  increased 
ability  to  incorporate  leading  edge  commercial  technology  and  evolution  of  requirements  during 
source  selection. 

Advantages  achieved  as  documented  in  the  case  studies  include  smaller  program  offices, 
protest  avoidance,  cost  control,  shortened  source  selection  schedules  and  improved  contractor 
performance. 

Risk  is  present  in  evaluating  offerors  on  the  basis  of  capability  statements,  specifically  in 
terms  of  past  performance  and  expected  capabilities.  Risk  is  also  present  in  the  potential  for 
inadvertently  transferring  ideas  developed  by  one  source  to  that  source’s  competitors.  There  is 
the  risk  of  increased  costs  associated  with  carrying  competition  into  further  phases  of  system 
development. 
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5.  Secondary  Research  Question  #5:  What  conditions  best  support  the  use  of  a 
multi-phase  source  selection  strategy? 

As  originally  stated  by  the  PBA  study  group,  there  are  three  main  indications  of  when  the 
multi-phase  strategy  best  fits:  when  there  is  a  high  degree  of  confidence  in  the  ability  to 
determine  a  fair  price  without  relying  on  supplier  cost  data;  when  the  requirement  is  fluid  and 
likely  to  change  significantly  after  exchange  of  information  with  potential  sources  and  when  two 
or  more  sources  are  expected  to  be  capable  of  meeting  the  requirements.  ” ' ' 

Other  conditions  noted  in  the  case  studies  that  support  the  use  of  a  multi-phase  strategy 
include:  experienced  persormel  willing  to  adopt  new  strategies  and  engage  in  revised  behavior 
patterns;  time  available  early  in  the  program  to  fully  explore  the  requirements  and  possible 
solutions  during  initial  phases  and  organizational  willingness  to  modify  regulatory  guidance  as 
needed. 


6.  Secondary  Research  Question  #6:  How  have  other  Federal  procurements 
overcome  barriers  and  utilized  a  multi-phased  source  selection  strategy? 

The  barriers  of  the  traditional  acquisition  approach,  regulatory  guidance  and 
developed  mindset  of  participants  in  the  process  were  overcome  in  three  ways.  The  Deepwater 
Project  used  specific  authority  to  deviate  from  agency  regulations.  The  Arsenal  Ship  program 
used  OTA  to  operate  outside  of  traditional  acquisition  methods  and  procedures.  The  JDAM 
program  used  DAPP  designation  and  blanket  waivers  to  deviate  fi’om  regulations.  The  mindset 
of  participants  was  overcome  by  focused  staffing  of  the  program  office  and  tailored  training  on 
philosophy  and  approach. 

The  barrier  of  organizational  resistance  to  change  was  overcome  by  concerted  efforts  to 
keep  higher  levels  of  authority  informed  and  through  education  and  staffing  efforts  similar  to 
those  used  to  overcome  the  entrenchment  of  the  traditional  acquisition  approach. 
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The  barrier  of  difficulties  associated  with  funding  was  overcome  in  the  JDAM  program 
by  demonstrating  cost  savings  throughout  the  execution  of  the  program.  Issues  surrounding 
funding-driven  schedules  were  not  overcome  by  the  programs  presented  in  Chapter  III.  In  the 
case  of  the  Arsenal  Ship  program,  funding  issues  proved  fatal  to  the  program. 

The  barrier  of  the  length  of  phases  was  overcome  by  modifying  schedules,  as  in  the 
Deepwater  example.  Attempts  to  overcome  the  length  of  phase  barrier  by  contractors  in  the 
Arsenal  Ship  program  resulted  in  concurrency  when  it  was  not  appropriate.  - 

The  bsLrrier  of  limiting  competition  in  follow-on  phases  was  overcome  through  the  use  of 
J&As  for  limiting  competition,  worked  in  advance  of  the  actual  request  by  the  program  office. 

The  barrier  of  requirement  and  solution  refinement  was  resolved  by  using  a  sole  source  or 
by  operating  outside  of  the  traditional  procurement  process. 

7.  Secondary  Research  Question  #7:  What  barriers  exist  that  preclude  NAVAIR 

from  fully  utilizing  a  multi-phased  source  selection  strategy? 

Barriers  that  exist  in  NAVAIR  include  timeline  restrictions,  the  usual  method  of  doing 
business,  procedural  guidance,  mindset  of  the  participants,  ability  to  evaluate  offerors  on  the 
basis  of  capability  and  appropriate  staffing  of  program  offices. 

8.  Secondary  Research  Question  #8;  What  should  the  underlying  NAVAIR 

procedure  be  to  fully  incorporate  multi-phase  source  selection? 

Appendix  B  outlines  proposed  procedural  guidance  to  use  a  multi-phase  source  selection 
strategy  at  NAVAIR. 
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9.  Secondary  Research  Question  #9;  What  are  the  pertinent  factors  for  screening 
potential  programs  for  utilization  of  a  multi-phase  source  selection  strategy  at  NAVAIR? 

Appendix  A  outlines  proposed  factors  for  screening  potential  programs  for  utilization  of  a 
multi-phase  source  selection  strategy  at  NAVAIR. 

10.  The  Primary  Research  Question:  What  are  the  essential  characteristics  of  a 
NAVAIR  source  selection  process  that  incorporates  to  the  maximum  extent  possible 
multi-phase  source  selection  as  recommended  in  the  1999  PDA  report? 

There  are  three  essential  characteristics.  First,  phases  are  used  to  fully  understand 
requirements  to  the  point  that  program  risk  is  significantly  reduced.  Second,  changes  to 
requirements  do  not  have  to  be  re-competed  and  may  occur  as  requirements  are  being  refined. 
Third,  no  J&A  is  required  for  other  than  full  and  open  competition  when  going  into  a  follow-on 
phase  considering  only  offerors  who  were  included  in  prior  phases. 

F.  RECOMMENDATIONS 

The  NAVAIR  SSPB  should  be  utilized  to  identify  potential  candidate  programs  in  which 
to  use  multi-phase  source  selection.  The  factors  presented  in  Appendix  A  should  be  used  as  a 
screening  tool  for  programs  considered.  Specific  authority  to  use  this  strategy  as  a  pilot  in  a 
program  should  be  sought.  Lessons  learned  firom  the  experience  gained  in  this  program  should 
be  documented  and  used  by  NAVAIR  and  the  DoD  acquisition  reform  office  to  fully  evaluate 
incorporation  of  regulatory  guidance  similar  to  that  provided  in  Appendix  B. 

G.  AREAS  FOR  FURTHER  RESEARCH 

1 .  Survey  pending  DoD  acquisitions  to  analyze  potential  for  success  with  the  multi¬ 
phase  source  selection  strategy  using  the  screening  factors  outlined  in  Appendix  A. 
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2.  Categorize  and  quantify  metrics  for  assessing  the  success  of  a  multi-phase  source 
selection  strategy  in  achieving  expected  benefits. 

3.  Assess  active  programs  outlined  herein  to  determine  whether  expected  benefits  at 
the  start  of  the  acquisition  were  achieved  by  use  of  the  respective  phased  source  selection 
strategy. 
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APPENDIX  A. 


PROPOSED  SCREENING  FACTORS  FOR  USING  MULTI¬ 
PHASE  SOURCE  SELECTION 


The  presence  of  the  following  factors  suggests  that  use  of  the  multi-phase  source 
selection  strategy  may  be  appropriate: 

1 .  A  high  degree  of  confidence  exists  in  the  ability  to  determine  a  fair  price  without 

relying  on  supplier  cost  data.  ”  ' 

2.  The  requirement  is  fluid  and  likely  to  change  significantly  after  exchange  of 
information  with  potential  sources. 

3.  Government  requirements  are  initially  stated  as  objectives,  or  in  a  broad  or 
performance  based  manner. 

4.  Potential  exists  to  take  maximum  advantage  of  commercially  available  technology. 
This  potential  can  be  in  the  form  of  first-time  use  of  commercial  technology  in  a 
defense  system  or  in  the  form  of  integration  of  commercial  and  military  technology  in 
previously  unused  combinations. 

The  presence  of  the  following  factors  would  help  ensure  the  success  of  a  multi-phase 
source  selection  strategy; 

1 .  Adequate  time  is  available  to  fully  accommodate  exploratory  phases  prior  to  or  at  the 
start  of  the  program.  Time  is  needed  to  fully  explore  the  requirements  and  possible 
solutions. 

2.  Adequate  commitment  of  funds  is  available  to  accommodate  potential  growth  in 
funding  requirements  during  early  requirement  development  phases. 
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3.  Two  or  more  sources  are  expected  to  be  capable  of  meeting  the  requirements. 

4.  Experienced  personnel  willing  to  adopt  new  strategies  and  engage  in  revised  behavior 
patterns  are  available  for  staffing  the  program  office.  Personnel  in  both  Government 
and  contractor  program  offices  need  to  be  taken  into  consideration. 

5.  Organizational  willingness  exists  to  modify  regulatory  guidance  as  needed  to 
accommodate  the  intended  strategy. 
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APPENDIX  B. 


PROPOSED  NAVAIR  MULTI-PHASE  SOURCE  SELECTION 


INSTRUCTION 


Discussion: 

The  multi-phase  source  selection  strategy  initially  selects  sources  determined  to  be  the 
most  qualified  by  evaluating  offeror  capability  statements.  The  capability  statement  includes 
limited  information  on  past  performance,  ability  to  perform;  conceptual  approach,  tuid' a  rough 
order  of  magnitude  price.  Government  requirements  are  initially  stated  as  objectives.  The 
Government  works  with  selected  sources  to  refine  requirements.  Major  consideration  is  given  to 
reducing  program  risk  and  taking  maximum  advantage  of  commercially  available  technology. 
Requirement  refinement  may  be  done  through  one  or  more  phases.  The  Government  may 
choose  to  fund  all  or  a  part  of  these  phases.  Requirement  refinement  continues  until 
requirements  are  imderstood  well  enough  by  both  Government  and  industry  to  allow  sources  to 
propose  a  firm-fixed  price  for  performance.  In  each  phase,  the  Government  may  elect  to  reduce 
the  number  of  sources.  In  the  proposal  development  and  evaluation  phase,  the  only  sources 
eligible  to  participate  are  those  selected  on  the  basis  of  their  initial  submission.  If  a  requirements 
development  phase  is  conducted,  sources  may  be  limited  to  those  participating  in  the  proposal 
development  and  evaluation  phase.  No  Justification  and  Approval  is  required  for  limiting 
competition  in  this  manner.  The  solicitation  outlines  the  extent  to  which  the  Government 
reserves  the  right  to  negotiate  requirements  or  terms  or  conditions  different  from  those  stated  in 
the  solicitation  without  amending  it.  Award  can  be  made  to  single  or  multiple  offerors. 
Following  receipt  of  proposals,  one  or  more  offerors  may  be  eliminated  during  the  evaluation. 

This  strategy  differs  fi-om  present  source  selection  strategies  in  two  ways.  First,  changes 
to  requirements  do  not  have  to  be  re-competed.  Changes  to  the  requirement  may  occur  as  it  is 
being  refined.  This  may  happen  during  a  contract  or  not  during  a  contract.  Second,  no  J&A  is 
required  for  other  than  full  and  open  competition  when  going  into  a  follow-on  phase  considering 
only  offerors  who  were  included  in  prior  phases. 
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Procedure: 


1 .  GENERAL  -  The  multi-phase  source  selection  strategy  may  be  used  to  procure  property  or 
services.  Single  or  multiple  awards  may  be  made.  Contract  award  is  predicated  on  obtaining  the 
best  value  to  the  Government.  This  is  a  competitive  process. 

2.  NOTIFICATION  -  Notification  shall  be  published  in  accordance  with  section  1 8  of  the 
Office  of  Federal  Procurement  Policy  Act  (41  U.S.C.  416)  and  subsection  (e),  (f)  and'  (g)  of 
section  8  of  the  Small  Business  Act  (15  U.S.C  637),  except  that  the  notice  must  only  include: 

a.  A  broad  description  of  the  scope  or  purpose  of  the  procurement.  The  information 
provided  has  to  be  sufficient  enough  for  potential  offerors  to  make  an  informed 
business  decision  about  participating. 

b.  A  description  of  the  basis  of  source  selection  as  outlined  in  either  subsection  three 
or  four  of  this  section,  depending  on  which  procedure  will  be  used. 

c.  A  description  of  the  information  offerors  must  submit.  This  may  include 
information  about  offeror  qualification,  the  proposed  conceptual  approach,  cost 
likely  to  be  associated  with  the  proposed  conceptual  approach,  past  performance 
of  the  source  on  Federal,  state  and  local,  or  private  sector  contracts.  Other 
information  that  the  head  of  the  agency  determines  is  necessary  to  select  sources 
to  enter  either  the  requirements  development  process  or  the  proposal  development 
and  evaluation  phase  may  be  included. 

d.  Any  additional  information  the  head  of  the  agency  determines  is  appropriate. 

3.  REQUIREMENT  DEVELOPMENT  -  The  head  of  the  agency  may  continually  develop 
agency  requirements  based  on  exchanges  with  sources.  As  part  of  developing  the  requirement, 
the  head  of  the  agency  may: 
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a.  Request  increasingly  more  detailed  information  from  sources  to  aid  analysis  of 
agency  needs  and  proposed  approaches  to  meet  those  needs. 

b.  Work  with  individual  sources  on  a  one-to-one  basis  to  improve  their 
understanding  of  agency  needs  and  the  acceptability  and  value  of  proposed 
approaches  for  addressing  agency  needs.  Agency  information  may  be  shared  one- 
to-one  with  individual  sources.  Information  shared*  in  a-  one-to-one  exchange  has 
to  be  shared  with  other  sources  participating  in  requirements  development  only  to 
the  extent  that  the  shared  information  is  necessary  to  propose  to  the  requirement. 

c.  Request  sources  to  develop  prototypes. 

d.  Enter  into  agreements  with  somces  to  fund  work  performed  to  participate  in  the 
requirements  development  process. 

e.  Request  sources  recommend  criteria  for  evaluating  further  participation  in 
requirements  development  or  participation  in  the  proposal  development  and 
evaluation  phase,  or  selection  to  receive  a  contract  or  contracts  awarded  under  this 
section. 

f.  Eliminate  one  or  more  sources  from  further  participation  in  the  requirements 
development  process  and  from  further  consideration  for  award  of  a  contract  imder 
this  section. 

4.  SOURCE  SELECTION  FOR  PROPOSAL  DEVELOPMENT  AND  EVALUATION  -  The 
head  of  the  agency  shall  make  the  final  selection  of  the  sources  that  are  eligible  to  enter  the 
proposal  development  and  evaluation  phase.  Sources  may  be  limited  to  the  number  determined 
to  be  in  the  best  interest  of  the  Government.  A  protest  is  not  authorized  in  coimection  with  the 
determination  not  to  consider  a  source  further  for  award  of  a  contract,  except  on  the  grounds  that 
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the  contract  awarded  is  outside  the  general  scope  or  purpose  described  in  the  initial  notice 
published  per  subsection  two  of  this  section. 

5.  PROPOSAL  DEVELOPMENT  AND  EVALUATION  -  The  head  of  the  agency  may  conduct 
a  competitive  process  in  which  only  the  sources  that  participated  in  the  requirements 
development  phase  are  eligible  to  participate  in  the  proposal  development  and  evaluation  phase. 
This  process  shall  use  the  following  procedures: 

a.  Each  source  selected  to  enter  this  phase  shall  be  provided  with  a  solicitation.  The 
solicitation  shall  state  the  basis  upon  which  a  contract  or  contracts  will  be 
awarded.  The  requirements  and  terms  and  conditions  included  in  the  solicitation 
may  be  stated  as  objectives.  Agency  requirements  may  continue  to  evolve  during 
this  phase.  The  solicitation  shall  provide  notice  to  offerors  of  the  extent,  if  any,  to 
which  the  Government  reserves  the  right  to  negotiate  requirements  or  terms  and 
conditions  different  from  those  stated  in  the  solicitation  without  amending  the 
solicitation. 

b.  At  any  time  during  the  development  and  consideration  of  proposals,  the  head  of 
the  agency  may  share  agency  information  with  individual  offerors  on  a  one-to-one 
basis  to  maximize  the  benefit  to  the  Government  of  each  offeror’s  approach. 
Agency  information  shared  with  one  offeror  must  be  made  available  to  other 
offerors  only  to  the  extent  that  the  shared  information  is  necessary  for  the 
preparation  of  the  proposal. 

c.  Following  receipt  of  proposals,  the  head  of  the  agency  may  seek  to  negotiate  with 
one  or  more  offerors  to  make  proposals  more  advantageous  to  the  Government. 
Offerors  may  be  eliminated  from  further  consideration  during  evaluation  and 
negotiation. 
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6.  CONTRACT  AWARD  -  The  head  of  the  agency  shall  award  a  contract  to  the  responsible 
offeror  with  the  proposal  that  is  most  advantageous  to  the  Government  More  than  one  contract 
may  be  awarded  if  it  is  in  the  best  interest  of  the  Government. 
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